Measure actual download speed

For the top few results it would be very useful to measure the actual download speed. The latency tells you the server is close and most likely has good routing, but it could still be taxed to near it’s maximum capacity.

To illustrate a simple test using the mirrors fetched from http://mirrors.ubuntu.com/mirrors.txt A list of mirrors, followed by the latency (5 pings) and the download performance as outputted by curl:

mirror: http://mirrors.noction.com/ubuntu/archive/ mirrors.noction.com : 11.75 13.36 14.32 14.36 15.04
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 54.0M  100 54.0M    0     0  15.8M      0  0:00:03  0:00:03 --:--:-- 15.8M

mirror: http://osmirror.rug.nl/ubuntu/ osmirror.rug.nl : - - - - -
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 54.0M  100 54.0M    0     0  32.8M      0  0:00:01  0:00:01 --:--:-- 32.8M

mirror: http://mirror.dataone.nl/ubuntu-archive/ mirror.dataone.nl : 12.74 13.81 17.42 83.72 14.00
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 54.0M  100 54.0M    0     0  24.0M      0  0:00:02  0:00:02 --:--:-- 24.0M

mirror: http://mirror.nl.leaseweb.net/ubuntu/ mirror.nl.leaseweb.net : 12.49 11.28 11.18 10.95 14.03
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 54.0M  100 54.0M    0     0  32.6M      0  0:00:01  0:00:01 --:--:-- 32.6M

mirror: http://mirrors.nl.eu.kernel.org/ubuntu/ mirrors.nl.eu.kernel.org : 195.34 189.46 192.50 190.03 189.61
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 54.0M  100 54.0M    0     0  7304k      0  0:00:07  0:00:07 --:--:-- 9196k

mirror: http://ubuntu.mirror.cambrium.nl/ubuntu/ ubuntu.mirror.cambrium.nl : 12.07 11.71 13.25 11.19 13.21
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 54.0M  100 54.0M    0     0  23.7M      0  0:00:02  0:00:02 --:--:-- 23.7M

mirror: http://ftp.tudelft.nl/archive.ubuntu.com/ ftp.tudelft.nl : 19.16 11.97 13.18 13.60 13.69
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 54.0M  100 54.0M    0     0  24.3M      0  0:00:02  0:00:02 --:--:-- 24.3M

mirror: http://ubuntu.mirror.true.nl/ubuntu/ ubuntu.mirror.true.nl : 17.14 13.80 15.07 22.22 9.96
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 54.0M  100 54.0M    0     0  33.4M      0  0:00:01  0:00:01 --:--:-- 33.4M

mirror: http://ftp.snt.utwente.nl/pub/os/linux/ubuntu/ ftp.snt.utwente.nl : 15.19 22.67 15.82 15.47 17.84
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 54.0M  100 54.0M    0     0  20.6M      0  0:00:02  0:00:02 --:--:-- 20.6M

mirror: http://ftp.nluug.nl/os/Linux/distr/ubuntu/ ftp.nluug.nl : 15.86 11.20 10.38 74.28 77.86
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 54.0M  100 54.0M    0     0  23.9M      0  0:00:02  0:00:02 --:--:-- 23.9M

mirror: http://mirror.i3d.net/pub/ubuntu/ mirror.i3d.net : 20.47 18.63 16.63 14.35 12.19
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 54.0M  100 54.0M    0     0  20.3M      0  0:00:02  0:00:02 --:--:-- 20.3M

mirror: http://nl3.archive.ubuntu.com/ubuntu/ nl3.archive.ubuntu.com : 21.67 19.40 18.24 19.69 19.01
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 54.0M  100 54.0M    0     0  31.8M      0  0:00:01  0:00:01 --:--:-- 31.8M

mirror: http://nl.archive.ubuntu.com/ubuntu/ nl.archive.ubuntu.com : 12.72 14.37 14.39 15.06 21.63
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 54.0M  100 54.0M    0     0  12.9M      0  0:00:04  0:00:04 --:--:-- 12.9M

mirror: http://mirror.amsiohosting.net/archive.ubuntu.com/ mirror.amsiohosting.net : 14.44 14.44 13.70 12.83 14.47
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 54.0M  100 54.0M    0     0  23.6M      0  0:00:02  0:00:02 --:--:-- 23.6M

mirror: http://mirror.1000mbps.com/ubuntu/ mirror.1000mbps.com : 13.05 11.30 11.43 11.41 15.09
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 54.0M  100 54.0M    0     0  33.6M      0  0:00:01  0:00:01 --:--:-- 33.7M

mirror: http://mirror.nforce.com/pub/linux/ubuntu/ mirror.nforce.com : 12.91 18.86 93.86 72.64 35.41
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 54.0M  100 54.0M    0     0  32.7M      0  0:00:01  0:00:01 --:--:-- 32.7M

mirror: http://mirror.transip.net/ubuntu/ubuntu/ mirror.transip.net : 14.20 16.50 11.64 14.43 24.58
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 54.0M  100 54.0M    0     0  17.4M      0  0:00:03  0:00:03 --:--:-- 17.4M

mirror: http://archive.ubuntu.com/ubuntu/ archive.ubuntu.com : 19.13 19.80 22.77 24.00 21.61
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 54.0M  100 54.0M    0     0  7752k      0  0:00:07  0:00:07 --:--:-- 10.6M

As you can see, several of the mirrors have really low latency but don’t offer fast downloads. For example, the nl3.archive.ubuntu.com server has a latency varying from 19.01 to 21.67 which would mean it doesn’t end up at the top of the list while the download speed of 31.8MB/s is far more important to me.

Similarly, mirrors.noction.com only downloads with 15.8MB/s with more favorable latencies (varying between 11.75 and 15.04ms).

for mirror in $(curl http://mirrors.ubuntu.com/mirrors.txt); do
    echo
    echo -n "mirror: $mirror "
    host=$(echo "$mirror" | awk -F '/' '{print $3}')
    fping -C 5 -q "$host"
    curl -o /dev/null "${mirror}/dists/xenial/main/installer-amd64/current/images/netboot/mini.iso"
done

[wolph]

I would like to propose a slightly different solution. Instead of limiting by size, limit by time. Just download for 1 second and see how much was downloaded in that time. For a slow connection it might only be 1 MB but for a fast connection it could easily be 100MB.

So with fast connections in mind, why would 15MB be too big? 100 Mbps connections are widely available in many countries these days and this script would also be useful for datacenters where gigabit connections are really common.

[jblakeman]

This is connected to #46 as it requires using http and ftp connections.