c# – parallel foreach c # and split collection

Question:

Is there any way to set a rule according to which the collection will be divided in parallel foreach c # between threads?

By default, if ParallelOptions.MaxDegreeOfParallelism is 4 and the collection consists of 100 elements, then the first thread will get the first 25, the second the second 25 elements, and so on.

Is it possible to do something so that the first one goes through 1 5 9 and t n element, the second goes through 2 6 10 elements and so on, the third goes through 3 7 11 and so on, and 4 goes through 4 8 12 and so on?

It seems that there is some kind of class, but it is quite complex in itself.

Can you provide examples that will allow me to do the behavior I want?

UPD 1:

In my task, a collection is a collection of file paths.

Each thread in parallel foreach takes one file at a time and feeds that file to Process.Start .

The division, which is described above, is desirable for the sake of processing the files in parallel within one directory, and then switching to the next.

Ie 4 streams first sit in one directory, and then go to the next.

This opens up additional possibilities for me: For example, since I know that the directory has been processed, I can immediately delete it.

Answer:

Such a rule cannot be set. But if the order in the output is important, then you can use the extension ".AsParallel (). AsOrdered ()".

var collection = new string[0];
var strings = collection
                  .AsParallel()    // параллелим 
                  .AsOrdered()     // хотим сохранить порядок
                  .Select(s => { return s; }) // обрабатываем
                  .ToArray();

As a result, we get a parallel processed collection with preservation of the order of elements. And then, if necessary, we go through the processed collection and select the necessary elements.

Other options (manual split) will give a performance drawdown.

There is another option to split the original collection into the required ranges and run parallel tasks:

var collection = new string[0];
var tasks = new List<Task<string[]>>();

tasks.Add(Task.Run(() => collection.Where(x => true).Select(x => { return x; }).ToArray()));
tasks.Add(Task.Run(() => collection.Where(x => true).Select(x => { return x; }).ToArray()));
tasks.Add(Task.Run(() => collection.Where(x => true).Select(x => { return x; }).ToArray()));
tasks.Add(Task.Run(() => collection.Where(x => true).Select(x => { return x; }).ToArray()));

await Task.WhenAll(tasks);

var result = tasks.SelectMany(t => t.Result).ToArray();

Everything will be calculated in parallel with the required criteria.

ps in .where we set the required criterion in .select calculate.

Taking into account the added clarification " UPD 1 ". We group the collection of paths by folders and execute each group sequentially:

var paths = new string[] { @"c:\test\test.txt", @"c:\test\test2.txt", @"d:\test\test.txt" };
var groups = paths.GroupBy(p => Path.GetDirectoryName(p)).ToArray();
for (int i = 0; i < groups.Length; i++)
{
    Parallel.ForEach(groups[i], path =>
    {
        // обрабатываем путь
        // Process.Start
    });
    Directory.Delete(groups[i].Key);
}

Or another option:

var paths = new string[] { @"c:\test\test.txt", @"c:\test\test2.txt", @"d:\test\test.txt" };
var groups = paths.GroupBy(p => Path.GetDirectoryName(p)).ToArray();
Parallel.ForEach(groups, group =>
{
    for (int i = 0; i < group.Count(); i++)
    {
         // обрабатываем путь
         // Process.Start
    }
    Directory.Delete(group.Key);
});
Scroll to Top