Question:
There was a simple download code:
internal static async Task<ImageFile> DownloadFile(Uri uri)
{
byte[] result;
WebResponse response;
var file = new ImageFile();
var request = WebRequest.Create(uri);
try
{
response = await request.GetResponseAsync();
using (var ms = new MemoryStream())
{
response.GetResponseStream().CopyTo(ms);
result = ms.ToArray();
}
}
catch (System.Exception ex) { }
if (response.ContentLength == result.LongLength)
file.Body = result;
return file;
}
I would like to add a download speed indicator. Google suggested that you can focus on the speed of reading the stream:
result = await CopyTo(response.GetResponseStream(), response.ContentLength, progressChanged);
private static async Task<byte[]> CopyTo(Stream from, long totalBytes, Action<DownloadProgress> loadedEvent)
{
var sw = new Stopwatch();
sw.Start();
var data = new byte[totalBytes];
byte[] buffer = new byte[81920];
int currentIndex = 0;
while (true)
{
int num = await from.ReadAsync(buffer, 0, buffer.Length).ConfigureAwait(false);
if (num != 0)
{
Array.Copy(buffer, 0, data, currentIndex, num);
currentIndex += num;
loadedEvent?.Invoke(new DownloadProgress(currentIndex, totalBytes, sw.ElapsedMilliseconds));
}
else
break;
}
sw.Stop();
return data;
}
To store information (and scroll up), I started a simple structure:
public struct DownloadProgress
{
public readonly long BytesReceived;
public readonly long TotalBytesToReceive;
public readonly long TimeMs;
public double GetSpeed()
{
var seconds = TimeMs / 1000.0;
if (seconds > 0)
return BytesReceived / seconds;
return 0;
}
public DownloadProgress(long received, long total, long time)
{
this.BytesReceived = received;
this.TotalBytesToReceive = total;
this.TimeMs = time;
}
}
In total, if you download in one stream, then GetSpeed at any time (except for the first second somewhere) shows the real speed.
The class that downloads the total figure eventually stores:
this.Speed = 0;
var file = await ImageFile.DownloadFile(this.ImageLink, dp => this.Speed = dp.GetSpeed());
this.Speed = 0;
Then I thought the easiest was left – I just added all the speeds at the top level and that's it:
return this.ActivePages != null && this.ActivePages.Any() ?
this.ActivePages.Sum(p => p.Speed) : 0;
In fact, it turned out very unpleasant behavior:
-
Most of the time the speed is actually displayed correctly.
-
The speed often jumps, and the spread sometimes exceeds the channel width. With a channel of 650 kb / s, the numbers jump from 300 kb / s to 5-8 Mb / s.
If the speed may well be lower at the end of one download and the beginning of the next, then exceeding the channel width is obviously impossible technically. Perhaps I missed the rounding somewhere, or is Stopwatch not accurate enough for such calculations?
UPD: the first suspicion was justified – the initial jump in speed spoiled the overall statistics. If the speed calculation method is done like this:
public double GetSpeed()
{
var seconds = TimeMs / 1000.0;
if (seconds > 0.1)
return BytesReceived / seconds;
return 0;
}
That speed is generally much more adequate, there are occasionally jumps above the channel width, but no more than 5%. You can increase the ignored time, then there will be no jumps at all.
There was a problem with the overall score. If for one stream the figure was reliable, then with multi-threaded downloads the figure often lies, the average of the indicator turns out to be lower than the real one (total time for the total volume).
UPD2: minimized all calculations, removed the structure, moved the logic to a static class:
public class NetworkSpeed
{
public static double TotalSpeed { get { return totalSpeed; } }
private static double totalSpeed = 0;
private const uint Seconds = 3;
private const uint TimerInterval = 1000;
private static Timer speedTimer = new Timer(state =>
{
var now = 0L;
while (receivedStorage.Value.Any())
{
long added;
if (receivedStorage.Value.TryDequeue(out added))
{
now += added;
}
}
lastSpeeds.Value.Enqueue(now);
totalSpeed = lastSpeeds.Value.Average();
}, null, 0, TimerInterval);
private static Lazy<LimitedConcurrentQueue<double>> lastSpeeds = new Lazy<LimitedConcurrentQueue<double>>(() => new LimitedConcurrentQueue<double>(Seconds));
private static Lazy<ConcurrentQueue<long>> receivedStorage = new Lazy<ConcurrentQueue<long>>();
public static void Clear()
{
while (receivedStorage.Value.Count > 0)
{
long dd;
receivedStorage.Value.TryDequeue(out dd);
}
while (lastSpeeds.Value.Count > 0)
{
double dd;
lastSpeeds.Value.TryDequeue(out dd);
}
}
public static void AddInfo(long received)
{
receivedStorage.Value.Enqueue(received);
}
private class LimitedConcurrentQueue<T> : ConcurrentQueue<T>
{
public uint Limit { get; }
public new void Enqueue(T item)
{
while (Count >= Limit)
{
T deleted;
TryDequeue(out deleted);
}
base.Enqueue(item);
}
public LimitedConcurrentQueue(uint limit)
{
Limit = limit;
}
}
}
As a result, when downloading, it is enough to report how many bytes were downloaded at the next moment:
NetworkSpeed.AddInfo(num);
And that's it, the NetworkSpeed.TotalSpeed indicator will display the average speed over the last 3 seconds. The average indicator as a whole has become more or less stable, although it slightly overestimates the indicators on my data. Well, it is obvious that if the thread pool is overloaded, then the timer will not work out in time and the speed will start to "jump".
Answer:
I am posting the solution that worked for me. The accuracy of the final figure is 95-99%.
public class NetworkSpeed
{
public static double TotalSpeed { get { return totalSpeed; } }
private static double totalSpeed = 0;
private const uint Seconds = 3;
private const uint TimerInterval = 1000;
private static Timer speedTimer = new Timer(state =>
{
var now = 0L;
while (receivedStorage.Value.Any())
{
long added;
if (receivedStorage.Value.TryDequeue(out added))
{
now += added;
}
}
lastSpeeds.Value.Enqueue(now);
totalSpeed = lastSpeeds.Value.Average();
}, null, 0, TimerInterval);
private static Lazy<LimitedConcurrentQueue<double>> lastSpeeds = new Lazy<LimitedConcurrentQueue<double>>(() => new LimitedConcurrentQueue<double>(Seconds));
private static Lazy<ConcurrentQueue<long>> receivedStorage = new Lazy<ConcurrentQueue<long>>();
public static void Clear()
{
while (receivedStorage.Value.Count > 0)
{
long dd;
receivedStorage.Value.TryDequeue(out dd);
}
while (lastSpeeds.Value.Count > 0)
{
double dd;
lastSpeeds.Value.TryDequeue(out dd);
}
}
public static void AddInfo(long received)
{
receivedStorage.Value.Enqueue(received);
}
private class LimitedConcurrentQueue<T> : ConcurrentQueue<T>
{
public uint Limit { get; }
public new void Enqueue(T item)
{
while (Count >= Limit)
{
T deleted;
TryDequeue(out deleted);
}
base.Enqueue(item);
}
public LimitedConcurrentQueue(uint limit)
{
Limit = limit;
}
}
}
How to use it – At the top level, we call Clear() when we start or finish a download, so that the results are independent of other downloads. In the place where the actual download takes place, we call the AddInfo method, indicating how many bytes we received in the next download cycle. You can use CopyTo from the header or DownloadProgressChanged from the WebClient . The main thing is to convey exactly the difference between the previous indicator and the current one.
The accuracy of measurements is provided by a timer ( System.Threading.Timer ), and therefore, in order for the accuracy of the readings to be reliable, the thread pool must be free to call the callback .
Well, of course, the result of all measurements is in the TotalSpeed property. If you want, you can add an event about its change, for timely display in the UI. The frequency of its update is specially synchronized with the timer – otherwise the figure changes too often and the user does not understand what the speed is.