Is Parallel.For/ForEach in C# actually worth it? (and how to modernize it)
Vložit
- čas přidán 22. 05. 2024
- Become a Patreon and get source code access: / nickchapsas
Check out my courses: dometrain.com
Hello everybody I'm Nick and in this video I am going to talk about the Parallel class in .NET and see how it performs compared to other methods and how we can modernize it for 2021. There are going to be intresteing gotchas about multi-threading and concurrency so buckle up and let's jump into it.
Async Parallel blog: / parallel-foreach-async...
Timestamps
Intro - 0:00
Parallel does not mean faster - 0:43
API benchmark for vs Parallel.For - 2:41
Adding Task.WhenAll in the mix - 7:40
Making the Parallel.ForEach method async - 11:11
Don't forget to comment, like and subscribe :)
Social Media:
Follow me on GitHub: bit.ly/ChapsasGitHub
Follow me on Twitter: bit.ly/ChapsasTwitter
Connect on LinkedIn: bit.ly/ChapsasLinkedIn
Keep coding merch: keepcoding.shop
#dotnet #parallel #csharp
Back in 2014, had a process with a lot of business logic converting a large dataset. Process took 36 hours to run on an i5, I improved and refactored the code a little to put it in a Parallel.ForEach on a dual CPU system that had 32 logical cores and an SSD, ran in 8 minutes. Parallel.ForEach saved a couple of my weekends!
I think you missed an important point in when Parallelism is better than Concurrency. Concurrency scales better when you are waiting for things to complete (typically IO), so you can yield threads to do other work while that waiting is happening instead of just sitting idle.
Parallelism (multi-threading) scales better when you have computationally expensive work where you cannot yield the threads back.
Was just about to comment this
Yeah exactly
parallelism is no synonym for mulit-threading! Most of the people have literally no idea how to distinct parallelism vs concurrent. First multithreaded just means that u have a system (like an OS) that allows you to create and handle multiple threads (contexts in a single program). This does not mean that you have multiple processors / cores that are able to parallelize them! But if you have multiple cores, the OS maybe decides to share workload among multiple cores with multiple threads. Having multiple threads (no matter if multiple cores availbale or not) means you are able to concurrently execute programs/work/workload/code on your machine (either by timeslicing or any other scheduling mechanism). To parallelize means: Sharing workload of an specific task/job/whatevername to multiple execution paths (whatever your system is able to)! This means if there are multiple cores, then your OS can share work amon those cores. If you have vector processing units -> then your OS can share work amon them, if you have grids (like with graphics-cards) -> same story, if u dont have any parallel facility, you are about to waste resources while calculating overhead for sharing resources. Further reading: Flynns taxonomy, Andrew S. Tanenbaum -> Modern operating Systems.
Mixing/matching them is fun as well. I'm working on a program right now that, at an extreme use case has to load 20,000 small text files, perform some computation based on their contents, then write the results out to a common text file for the entire job. I started out using the Parallel library, some rewrties having multiple stages with BlockingCollection queues in between. However I've found the Dataflow library with its data blocks to be much more efficient. With Parallel, the best I was able to get (with significant benchmark testing and trials on how much each stage gets Parallelized) was 12 seconds to process the whole job. With Dataflow, it went down to 8 seconds.
I think the video proves this point in the second example
id like to add, the reason that running parallel operation on an array or int is slower than regular foreach has to do with how the runtime and system (memory) handles synchronisation of an array thats getting updated across multiple cores.
on single core, the cpu loads the array on a single core, goes to L1 memory and quickly iterates through it. If multiple cores are involved, cores that work on the same “block” of the array needs to sync with other.
an optimization is to break up the array in blocks and parallelize based on those blocks. the size of a block is based on the size of the cache line, if i remember correctly
So do you suggest using a Span or ArraySegment?
Dude, you just keep knocking out of the park with these videos! Keep up the good work man!
I did a terrain generator in Unity some years back. I implemented a hydraulic erosion system there. Essentialy what happened in the system was that it simulated raindrop flowing across the ground. On a rather small map you simulate about 50k-100k raindrops. Parallel.foreach saved alot of time. Though ultimate performance boost came from converting the algorithm into a compute shader.
i guess it is because the shader runs on GPU, which is designed to run parallel work, whereas the algo was running on CPU?
Great video Nick, thanks for covering this centrally important topic! And it illustrates precisely how to objectively quantify its results relative to the synchronous (single-threaded) alternative.
.NET 6 is adding a Parallel.ForEachAsync method
Spoilers! Yeah jokes aside the original draft for this video was done before April 8th which is when we found out about Parallel.ForEachAsync, but since it is not GA yet, I didn't wanna include it. Looking at the issue here github.com/dotnet/runtime/issues/1946 it basically is what I am doing in the last example in this video but without the AsParallel call.
So What will be the diffrent between ForEachAsync and whenAll?
I haven't seen the source code for the latest implementation but the one I had seen back in 2018 when they started talking about it basically did what I do in the last example I showed in this video. The only thing that (I think but don't quote me) it doesn't do is use AsParallel. You can check it out here: github.com/dotnet/runtime/issues/1946
Broooooooooooooooooooooooool
@@nimrod2142 you have a point but this is needed fam, that function is needed and it makes life much more easier.
That's a really great video.
For future readers I would like to point out that whenall and parallel can not always be used interchangeably, as also described on previous comments.
Top drawer content! Learnt so much!!! You’re a champion
Great video. I think that you should have used a concurrent collection in the AsyncParallelVersion benchmarks, which would affect the speed of the tests. Also I think it would have been convenient to mention that the order of the results is not preserved.
Pretty good CPU... yeap... 5950x is pretty good... not good enough of course but "pretty good".
Well, I got the same and can confirm that it is indeed pretty good.
I used a Parallel system for parsing large log files. This was back when you needed to assign to Tuples and do several things manually. It works best when you actually do processing on a large numbers of resources...
Awesome! I’ve been looking for an efficient way to do parallelization for a project.
Very good info. Thanks
Nice comparison on WhenAll & Parallelism. It's very interesting WA is more efficient.
Thank you Nick, before I resolved similar task through use simaphoreslim, but today you teached me how I can do the same with high level approach and my code will be cleary)
Niceeeee, I needed this for a project I am working on.
Take that Geo Differentials!
This is wrong. The async version does ToList(), the parallel version does List.Add() in the body of a Parallel. The Add() method is not thread safe... So I think the async version will allocate more and bigger arrays... Async and parallel version will produce different results...
I'm planning a CPU heavy application. This may turn out to be very useful. Thanks.
I often use `WhenAll`.
When I know the list is going to be very big, and tasks might take a long time to complete, I will add a semaphore to limit the number of tasks that will actually be running at one time.
Thanks!
Nice video. It all depends on the usage, my general rule to start with: Parallel for CPU bound operations, Task (async) - IO-bound.
Hey Nick, such a great channel and video, first of all. Thanks for that.
Just wanted to ask: Should we always give maxdegreeofparallelism? I have an application that is running on kube, so the number of max core may vary. What kind of precautions should I take?
You could try another version of the AsyncParallel - can await Task.WhenAll instead of awaiting each task one by one. Not sure if it’ll be faster or might increase memory consumption, though. But it would be interesting to compare
👍🏽 superb.
Instead of a Partitioner to you could also have used a Semaphore with 'await _semaphore.WaitAsync() ' in the executed task to limit the thread count with asyncTask.WhenAll().
I have been looking for something similar like executor service in java, which I think the async parallel foreach would help and resolve. Another way I implemented async parallel for each was using Bulkhead policy in Polly with the httpclient. This is only when I want to parallelize dependency http calls.
Have you visited the IAsyncEnumerable interface and await foreach in any video?
Since .NET 6 there is now a Parallel.ForEachAsync which I suspect might do similarly to what Nick implemented himself here!
Personally, I find the best place to utilize parallelization is when you are working with BIG DATA. In my experience, that might be grid's of data, like an image. Or just any data that you know is going to be big, ane therefore slow things down by that virtue. So I was glad that I found Parallel.For a while back.
For some reason thoough, Parallel.ForEach seems to run more slowly than Parallel.For. I guess because it has overhead due to it being more generalized?
Either way, before I discoved the Parallel class, I was basically trying to find a simple way to process big blocks of data with Task and all of it's stuff. But it seemed like doing that didn't make a lot of sence. Because each task also allocates memory. And for large grid, I guess that would add up a lot.
So like, if I had a Task for each pixel, the list of tasks I would supply to "Task.WhenAll" would be huge! So I never even tried that.
If anyone decides to read my ramble, I'm just hoping you (somone) might have somthing to add to this. Becuase I don't think I know a LOT about different multithreading technique in C#, so I can use the best method for my cases.
ForEach is slower than For in general. Might add up per thread used in the parallel version so you notice it more maybe?
Hi Nick, Is it possible to make a video about the best method to create a multithread/multiworker application?
Parallel is bound by the number of threads a computer has and assumes that every thread will pickup a similar workload. It also isn't very efficient when it comes to memory usage.
In my experience, when I use multiple 'threads', there is a huge imbalance of work between the threads. You'll have a few threads with high usage and others fairly low.
I usually get better results if I simply create Tasks and allow the OS to decide how to execute them for me.
Yes , have one customer complain that a button freezes the program for a second, checkmate.
You pronounced it correctly. Parrarraaallermumal
Can you make a video about adding W3C TraceContext using dotnet core’s built-in Actions?
Could you do a video on the 'new' .Net 6 Parallel.ForEachAsync method or does it behave just like 11:36?
Nick, great video, and I have a question.
If we're using EF Core and we try this method on some tasks that use the same db context, would it be an issue? I know task.WhenAll can present some issues and that is basically what we're doing, but what about the parallel .ForEach, does it have the same effects?
Thanks.
DbContext is not thread safe, so yeah, you might face issues eventually. Indeed, EF should be throwing you an exception if performing such kind of operations.
Hi, can you please showcase how to do CRUD with ef core using parallelism, because ef core throws a lot of errors when doing db operations under parallelism ?
Thanks
Good video! The Parallel class has been bugging me for quite some time now, since it makes the impression that it's great for parallelism upfront, but it's just a super awkward thing for naturally async programs.
Its not awkward in the slightest. It is by far the simplest way to extract tens of times more performance from a given system for the right workload. All the workloads he showed here are not at all what it should be used for, why anyone would even attempt to use it for that I have no idea.
It is for making compute bound workloads parallel, that is all.
From the sounds of your comment it seems like that is not something you need for your atypical workloads, which is fine but that doesn't make the Parallel class bad.
Muy bueno, una consulta como sacas esos tiempos o usas alguna librería?
Usa benchmarkdotnet, tiene un video de esa librería :)
@@hernanar3647 gracias
Does this work in a azure function hosted service? If I'm not mistaken, azure function is billed by calculation time * run count * rate. I don't think it says something about cpu usage (since it is a serverless PaaS platform). If yes, we might just "cheat" Microsoft
4:53 - did you mean "vast majority" instead of "fast majority"?
If it wasn't for AMD we'd still be stuck with 4 cores ;)
You should be aware that List is not thread safe, so using it with Parallel call is not a good idea. You could use ConcurrentBag. Major difference is that ConcurrentBag is an unordered collection, therefore so items are not ordered nor accessible by index.
I am not testing the consistency of the data but the raw output of the approaches. Introducing a thread safe construct would impact the test results due to locking, which is not what I was testing for.
There are 2 concrete uses for Parallel.ForEach:
1. Running multiple, similar processes (taking each image in a folder and rotating it 180 degrees)
2. Distributed queries (calls to multiple rest api's)
Sorry but both of those are I/O-bound and will be much more efficient and probably faster running concurrently with TPL. Parallel is for CPU-bound operations that do not benefit from asynchrony. Image files can use async streams to read and write files and REST calls are the epitomy of async task. With Parallel.ForEach these will spinwait the CPU cores most of the time doing nothing but generate waste heat.
@@the-niker Rotate 1,000 images in serial and then in parallel and tell me which is faster. Then do the same thing with 1,000 calls to disparate URIs.
@@fudhater8592 You misread my comment, I never said not to use parallelism. I said to use TPL = Task Parallel Library instead of Parallel.ForEach for anything that can be done asynchronously. You will use less CPU power to do the same work, therefore your server can serve more clients at once and scale better. Parallel.ForEach will spin CPU in place at 100% core utilization when it's waiting for I/O to finish.
significant that the early tests didn't max the 32xCPU showing a significant [external] I/O / network / other-resource reason. Last one (albeit the actual webapi engine) stressed all cores to good advantage. As ever: test, test, test and YMMV !
The initial Parallel one was actually pushing it to 100% when I was running it for sustained load but 50-60% of that CPU usage was the Web API server running in the background and not so much the consumer app. This actually shows how well .NET can handle multiple requests at the same time from a consumer perspective.
Why you didn't test Parallel.ForEachAsync() from .Net 6?
Hello, man. I implement your solution (talking about ParallelForEachAsync) and find out that is a leak somewhere in this code. Every time, when I'm running some tasks, similar one to another, on 10000 examples I'm getting something around 150mb allocated memory in process that doesn't cleaning.
Could you please check it out? If it's still not out of date solution and you have time and patience for that ofc :)
P.s. Ofc if you still reading comments in past videos :D
Hmm, I'm not sure whether list access is properly synchronized in the AsyncParallelVersion.
It is not but for the purposes of this video is doesn't matter. I am not testing data consistency but performance. If i introduced locking then it would be an unfair test since not every parllel operation requires thread safety. Imagine completely independent tasks.
@@nickchapsas Thanks for your repy, I see. On the other hand, efficient accumulation of the results is sometimes a hard problem by itself, and I think the unaware watchers should at least know that your code cannot be put into production without solving that problem.
Pretty sure that calling a simple api endpoint to get the same value 1000 times isn’t something that anyone would put into production. Can’t account for every potential misuse of the content
Could you make a video about garbage-less async programming? :) I wonder if this sorcery could be used in gaming.
What do you mean with sorcery? Parallelism has been used in gaming for many years already
Your video are very imformative and you write your code very clean and in a professional manner !!
I have a video request for you ;)
Would you please create a video about thread-safe / unsafe collection with some example :)
You did use List for Parallel class , i'm really confused , I think it should be ConcurrentBag
Hello Nick,
Thank you for your videos!
I started to learn C# one year ago. I want to be a solid C# Developer. and maybe after 3-5 years become an MVP.
Most of the time I read documentations and watch videos. but I still feel like there is something missing. So I decided to start reading the following books in order:
- C# 9.0 in a Nutshell: The Definitive Reference 1st Edition
by Joseph Albahari
- C# 9 and .NET 5 5th Edition by Mark J. Price
- C# in Depth: Fourth Edition 4th Edition
Am I moving on the right path? Do you recommend other books to start with?
I am not surrounded by an experienced C# (Microsoft Technologies) Engineer. That's why I am asking you to guide me or check with me every few months.
Thanks again.
My question would be this: Are you using what you're learning? Are you building stuff? What are the problems that you have that you can use those skills to solve and refine?
I believe that one doesn't truly learn unless they do.
@@nickchapsas I keep building applications and apply what I learned. I built WPF, Blazor, Winform, asp.net MVC apps. Even if cannot apply what I learned in work I build a new personal project.
Once I had a project that needs to communicate with the desktop app(QuickBooks) through COM and then communicate with Web App. so i had to use Azure service bus because.
For the new books, I am planning to build a new application and keep practicing on it.
I totally agree with you that we should learn and do. But sometimes when I work on a project I keep focusing on solving the project problem.
My goal is to learn the whole C# as a language so when I face a new problem or design a new project I be able to use the whole language's power and capabilities.
(I am a full-stack engineer with 4 years of experience, but when I change the job the technology stack changes. so my experience is bread more than depth) so my goal now is to have a depth experience.
@@AhmadRadi legacy stuff bro. You should take look at ASP. Core. Only Blazor is still alive.
@@n3kras0vh Legacy stuff is dead? Blazor is new and it could end up like Silverlight. Microsoft if big corporation. WinForms is used, sometimes WPF. Do you believe that Ahmad will be working on only very modern projects?
I think, that Ahmad should know (depply or shallowly) many solutions.
There is not a lot of sense to compare a lot of small independent operations. Usually, it is better to run complex operations in parallel, especially when they have different execution times. The best case is when these operations have cross-dependencies with each other.
Now if I can make this work with Entity Framework Core! Maybe someone can point me to a good example.
parallel may be hard for you to pronounce, but take solace in the fact that it's equally hard for me to spell correctly
The modernized version isn't really a benefit unless your loop is doing something with async aspects tho, is it? If all you're doing is crunching some data (for example image processing), there isn't really anything asyncable about that unlike your example where you are waiting for APIs to respond to something.
Why do I always think once the video starts, you'll start speaking in spanish :D
First woohoo
A big flaw in your understanding and/or methodology here is that Parallelism is good for compute style tasks, crunching numbers and that sort of thing. It is not in any way recommended for IO or Network heavy tasks.
The Parallel class when used correctly is a trivial way to get a 16x or more speedup (on your hardware), try doing some real work on some data and then you will understand.
Parallelism as a concept and how you make the most efficient use of it are two completely different things. Sure heavier workflows will benefit more from a parallel operation but that’s not what I’m testing here
In the parallel version of youtubeSubsribers, you don't have to blocking a thread with get result. Why are you thinking about awaitable add list when you can await the task a then add to list?
Parallel.For(0, count, async I => {
var result = await subsriberTask;
list.add(result);
})
And you are saying "it will not wait for the thing to do to actually run..." Await is not about to run the task. It would not wait for the task finish.
Because that’s not possible. The Parallel.For has no asynchronous overloads so if you do what you show in your code you just messed up your app’s logics. The runtime will treat it as an event because it accepts and Action so it will act completely differently and not await it properly
@@nickchapsas this code is valid. I can try it once again, when I will be at home. If parallel for not have func task variant, it would end up as error. Why would compiler convert func task to event? Can you send me some info about it? Or show it in the decompiler?
@@filipweisser2634 Because it is the same implicitly. The code is valid but it is an Action not a Func. That's why they added Parallel.ForEachAsync in .NET 6. The code snippet is valid but what you/it is doing is wrong.
Concurrent = Do More With Less, Parallel = Do More With More 😁
Now say "Rural Juror"
Para-lel.
Could we say you tried to DDOS yourself? Because this surely became a stress test for the local API😂
czcams.com/video/lHuyl_WTpME/video.html
You got here in far worse issue than context switching. This code will for certain end with False Sharing, you could benchmark this with "cache misses" metric. Despite the fact that for simple int array iteration loop unrolling would do better job, but If you would chunk this collection on a Cache Line boundary it would execute incomparably faster.
Would you do an updated version to the compare all versions of Async Parallel For Each to the Out of the box available Parallel.ForEachAsync(...)?
I also have found a version that might be an updated version on the one mentioned on this video.
using System;
using System.Collections.Generic;
using System.Threading.Tasks;
using System.Threading.Tasks.Dataflow;
...
public static async Task AsyncParallelForEach(this IAsyncEnumerable source, Func body, int maxDegreeOfParallelism = -1, TaskScheduler? scheduler = null)
{
ExecutionDataflowBlockOptions executionDataflowBlockOptions = new ExecutionDataflowBlockOptions
{
MaxDegreeOfParallelism = maxDegreeOfParallelism
};
if (scheduler != null)
{
executionDataflowBlockOptions.TaskScheduler = scheduler;
}
ActionBlock block = new ActionBlock(body, executionDataflowBlockOptions);
await foreach (T item in source)
{
if (item != null)
{
block.Post(item);
}
}
block.Complete();
await block.Completion;
}
I've used this code when the function is not O(1):
Parallel.ForEach(Partitioner.Create(enumerable, EnumerablePartitionerOptions.NoBuffering), new ParallelOptions() { MaxDegreeOfParallelism = -1 }, action);
Obviously doesn't account for IO, but has made some of my code faster when it is not the exact same speed the whole way through.