How IEnumerable can kill your performance in C#

Isn't too hat obvious for any who got to the stage of using enumerables??

Great and very important point Nick

why is it that he can write a method without any enclosing class?

The yield in the the example does not have the same behavior as returnn list.Select()

Basically, IEnumerable has deferred execution and executes again if the result wasn't stored in memory, just an iteration

Example too simple, can not apply to production code. I don't understand.

Also, for this specific example, or any time you want to return some kind of finite collection on which you want to be able to enumerate and get a count, you can return a ICollection.
The adventages to returning an interface type is to be able to change the underlying collection if needed without impacting the usage of the method.
For example, if eventually you have work to do on each entities and want to parallelize it, then you might want to use a ConcurrentBag instead of a List, but both would satisfy the ICollection signature, so no refactor is needed for consumers of the original function.

me putting .ToList() everywhere to prevent enumerables from behaving unpredictably: there. now I should be safe.
Garbage collector: *whining in agony*

I havent watch vid yet Nick "Epic" Chapsas is a Legend

Thank you for tip, quite interesting.
One question.. Does your courses come with certificate at conclusion?

I often return IReadonlyList and sometimes arrays. Most of the time, I do not need Lazy Loading

it never come to me that method enumerable behaves same way, but indeed it does. Nice stuff ty!

Calling ToList or ToArray is forcing a multiple enumeration. You enumerate to create the array and then you'll enumerate it again to process it. Try to avoid using ToList/ToArray when you can. You only want to iterate any given list once. The count could have been computed by the GetCustomer and returned as an out param.

I do actually know of this, and typically do take this into account; I'd be lying if I said I catch myself [or others] every single time :). It's one of those things you miss if you're working fast

I've switched to using IReadOnlyCollection or IReadOnlyList in most cases. The only time I use IEnumerable is when I don't need/want all items to be in memory at the same time, or if there could be a reason to only enumerate some of the items. If I had a CSV with 1,000,000 customer names and I wanted to know how many Nick's there are, I could read the file line by line, check if the name is Nick, increment a count, and move to the next line without storing all the names. Or if I wanted to get the address of the first 5 Nick's in the file, I could enumerate till I find the 5 Nick's, and then stope enumerating.

no

😆

Thanks for this video. I don't use IEnumerable. After that video i'll still so :) but i learn why i'm not.

Ive know this for awhile and I always thought it was a problem with .Count(). I have creatively coded around .Count() in many projects.

I don’t have resharper and don’t know about this until now…damn

I love that JetBrains catches possible multiple enumerations
BUT OH MY GOD If you don't enumerate the same IReadOnlyList multiple times it will NAG You endlessly to change to parameter to IEnumerable
Which is annoying because your parameter already conforms to IReadOnlyList then it will again nag you to change it back when you enumerate one more time

IEnumerable can kill your performance in C#, but if you want to be 100% sure, combine IEnumerables with databases and Lazy loading features. Devastating.

thanks man u r the best!

You have no idea how much this helped me today! I was looking at a problem where counting an IEnumerable with zero elements in it resulted in a significant delay and I thought I was going crazy! I had no idea that IEnumerable would be lazily evaluated. Thanks for the help! :)

I knew about that and also felt into the downsides of it. Since then I am cautious when I get an IEnumerable and check my call stack if it is used multiple times (aka enumerated).
But also I remember to have read in the official c# best practice guide to use IEnumerable as return type and parameter. (did not looked it up again).

This misunderstanding comes from the fact that enumerating with "yield return" statements effectively turns the method into a coroutine. Once the end of the method is hit, it is considered enumerated and local variables are deallocated. Then you start enumerating it again and it has to redo everything.

Oh LINQ and IEnumerables, such a blessing and a curse. Very easy to abuse and misuse.

I been programming with C# for about 15 years and there are parts about it that still mystify me. Your example of obtaining a count via an IEnumerable reminded me of how I learned on my own a similar situation with your example. In my case I was loading over 100k records. EF was new to me and I couldn't understand why my app was taking a performance hit until I discovered the difference between IEnumerable and IQueryable. From then on it forced me to take into consideration the overall purpose of the program and how to use IEnumerable properly. You are very well versed in the programming language, more than me after working with C# for so long.

On a side note, back when I was learning programming in 1991 I asked a senior developer of our mainframe why people are sloppy with their code. He told me that it will only get worse because as computers get faster it will compensate for bad coding practices and the end result will be lazy programmers. I came from learning to program on a mainframe environment where every byte counted. We ran accounts payable and payroll for 300 employees. All of it was done on a 72 megabyte hard drive.

The general advice for any caller of iterators that return IEnumerable<>, is to not mix cursor calls like ForEach, with aggregate functions like Count if both results are in scope of each other, unless the first call casts the result to a collection or array and the second call uses that cast result instead. Not doing that is not just a matter of performance; that's even potentially the least of our worries. The problem is instead that most likely we just introduced a potentially hard-to-find bug if the source data can be changed by a third party between both calls. But if both calls are not related and they are out of scope of each other, do not cast. That's a potentially expensive operation in itself.

When you switch to .Select(), the file is only read once while each line is selected twice; and then you could just append .ToList() to the .Select() to return a list of Customer's without splitting each line twice.

Simply call .ToList() and problem solved.

I really like your videos. This one was good too but the title was not really correct. Your presentation makes it look like the problem is IEnumerable. Its the correct way to return something when you don't want to limit the implementation of the method to a specific collection type. In fact, the problem is the Select() and knowing when to use Select().ToList() . In fact, you didn't even use ToList() instead you used a clunky list.Add() method. Also Count() is to be avoided in some cases. Length and Count are preferred (but hopefully Count() uses Length and Count behind the scenes, but it still requires a method call and an if statement). Another thing to realize is that by returning a LinQ QueryCollection type instead of IEnumerable you can avoid the double iteration? In that case, your Title would have been more appropriate. Whatever, I enjoyed the video anyway. Your presentations are always top notch in quality !

Resharper taught me, now i understand it

good to know, did not know before

How do you gain all this knowledge regarding perfomance?

The beauty of IEnumerables is lazy/deferred execution.
A trap (per this video's message) if you don't have a grasp of what it is.

Lazy/deferred execution I believe was borrowed from the Functional paradigm.
The idea is that you have a set of logic/algorithm which wont be executed/evaluated
unless with explicit intention.
In C# LINQ, you express the 'intention' by calling operators like
.First()
.ToList()
.Count()
.Any() etc.

Examples of lazy LINQ operators,
.Where()
.Select()
.OrderBy() etc.
These return an IEnumerable of <T>.
Lazy/deferred execution shines when composing/chaining functions and
when you intend to use your functions in between a "pipeline". Hence the above 3 are often used in a query chain/pipe.

Pertaining to collections, lazy evaluation passes only 1 item to each node/operator in the chain/pipe at a time.
But for eager evaluation, the whole collection is evaluated and passed down.
If there were conditions of 'early breaks', the latter won't benefit as the collection has been prematurely evaluated.
E.g. a lazy pipe/chain
products
.Where(p=> p.InStock()) // each product 'in stock', will flow down..
.Where(p=> p.Price < 3.14) // but only 1 at a time and not the full list because 'where' is lazy.
.Select(p=> p.ToShippable()) // Concatenated lazy chains act and behave as one (select is also lazy).
// I often combine multiple individual lazy operators to solve complex problems with very little concern for performance penalty.
// Shifting the order of the operators around is also quite easy as they are somewhat stand alone..

That IEnumerable kinda looks like a JavaScript Generator Function. ;-)

Anytime I see IEnumerable<T> my brain just says that results will be streamed one at a time when I ask for them. That has helped a lot.

However multiple enumerations is one of those things that have been around for decades and shows the perpetual state of inexperience our industry as a whole is in. Async/await and the TPL is another.
No matter how long async tasks are around we'll be seeing it used wrong all over the place.

The count method on IEnumerable introduces cockroaches not bugs. It shouldn't exist for enumerables. I wonder who's the stupid ass implementing this method.

Totally didn't know about it. Thanks Nick.

why would you ever need to return ienumerable anyway

yay - I was able to find the issue before Nick pointed it out. =) Though, I only looked for it because Nick pointed out there was a problem though...probably would not have cought it in real life.

As an old programmer I must say I don't like those modern ideas... gross

I did not know/understand this, but now I totally do. Thanks!

Why does no one have this problem with `Iterator`/`IntoIterator` in Rust? 🤔

In my opinion, all this is obvious.

Thanks for sharing this, didn't know about it, but I personally never use IEnumerable. However looking at colleagues code during code review I can now point this issue out to them when I spot it. Cheers.

one interviewer asked me to compare IQueryable and IEnumerable :)

I can't say I have ever seen a yield return of an IEnumerable in 15 years of C#. Return of an IEnumerator in coroutines, yes, but not this. Must be a business software thing.