c# – How to loop again in Parallel.For?

Question:

Let's say I'm downloading pictures in 100 threads. If any iteration caused an exception – how to repeat it again?

Parallel.For(0, newLst.Count, new ParallelOptions { MaxDegreeOfParallelism = 100 }, (i) =>
      {
           DownloadImage(newLst[i]);
      });

Answer:

Do DownloadImageAsync , which is not in Parallel.For , but simply by limiting the number of simultaneously executing tasks, for example, using this code:

    public static IEnumerable<Task<TTask>> ForEachAsync<TItem, TTask>(
        this IEnumerable<TItem> source, Func<TItem, Task<TTask>> selector, 
        int degreeOfParallelism)
    {
        Contract.Requires(source != null);
        Contract.Requires(selector != null);

        // We need to know all the items in the source before starting tasks
        var tasks = source.ToList();

        int completedTask = -1;

        // Creating an array of TaskCompletionSource that would holds
        // the results for each operations
        var taskCompletions = new TaskCompletionSource<TTask>[tasks.Count];
        for(int n = 0; n < taskCompletions.Length; n++) 
            taskCompletions[n] = new TaskCompletionSource<TTask>();

        // Partitioner would do all grunt work for us and split
        // the source into appropriate number of chunks for parallel processing
        foreach (var partition in 
            Partitioner.Create(tasks).GetPartitions(degreeOfParallelism))
        {
            var p = partition;

            // Loosing sync context and starting asynchronous 
            // computation for each partition
            Task.Run(async () =>
            {
                while (p.MoveNext())
                {
                    var task = selector(p.Current);

                    // Don't want to use empty catch . 
                    // This trick just swallows an exception
                    await task.ContinueWith(_ => { });

                    int finishedTaskIndex = Interlocked.Increment(ref completedTask);
                    taskCompletions[finishedTaskIndex].FromTask(task);
                }
            });
        }

        return taskCompletions.Select(tcs => tcs.Task);
    }

Now you can do this:

var tasks = newLst.ForEachAsync(n => DownloadImageAsyncWithRetry(n, retryCount), 100).ToList()

And just shove the logic of retraining into DownloadImageAsyncWithRetry :

public Task<Result> DownloadImageAsyncWithRetry(input)
{
  var tsk = DownloadImageAsync(input);

  // Тут все зависит от того, каким именно образом определяется неудача.
  // Если это исключение, то вешаем ContinueWith, если это код возврата,
  // то проверяем его и пробуем повторить запрос.
}

Yes, Parallel.For is not a good idea in this case, since it is intended primarily for CPU Intensive operations, but here it is clearly IO Intensive. In this case, it is logical to make the method itself asynchronous, which will pull the asynchronous API for loading pictures.

Z.Y. Take the ForEachAsync code from here .

Post Views: 12
Scroll to Top