We are interested in validating/comparing the PingER derived throughputs with those obtained by the Ookla Speed test measurements. I therefore contacted Ookla by email on November 27, 2007 and Mike Apgar and Doug Suttles of Ookla were kind enough to reply, answer questions, and on December 6, 2007 provide data suitable for importing into Excel (CSV format) and analyzing.
Ookla Speedtest.net Information
Ookla maintains a fleet of speed-testing servers all over the world. A user chosen server downloads a file of incompressible data to the user's client and then uploads it. They calculate the throughput by knowing the amount of data transferred and then dividing this by the time taken to transfer the data in each direction. A summary of this data is provided to the client user in real-time showing average download speed, upload speed, and ping Round Trip Time (RTT).
...
Ookla then uses the 95th percentile speeds for the IP address of each client. This way if a connection speed improves over time (or a better testing server for the region is added), it will improve the overall results. The same can be said for degradation of speed. These 95th percentile speeds are then what are averaged to determine the speeds shown in their data set. The ip_addresses column is the number of unique IP addresses from a specific country that have taken a test at Speedtest.net. The file provided contained aggregated data with one line for each country. It included data from 174 countries. The columns contained: the Top Level Domain, the Country name, the number of IP addresses, the average donload speed in kbits/s, the average upload speed in kbits/s. The data for each country was from >= 100 IP addresses.
Results
A rough summary of the results shows: US has most IP addresses = 5.7M. Highest download speed is Japan=11Mbits/s, followed by Sweden, Latvia, Romania, Netherlands, Bulgaria, Singapore, Germany, France, US. Slowest download speed is Cameroon (199kbits/s) followed by Mauritius, Botswana, Angola, Madagascar, Nepal, French Polynesia, New Caledonia, Tanzania, Kenya (i.e. bottom 5 in Africa). Fastest upload is Japan (5.1Mbits/s) followed by Russia, Romania, Sweden, Bulgaria, Latvia, Hong Kong, Lithuania, S. Korea, Moldova. The large number of Eastern European countries is interesting and not understood. The slowest download speeds are Madagascar (70kbits/s), followed by Yemen, Angola, Cameroon, New Caledonia, Botswana, Uruguay, Nepal, Zambia, French Polynesia (276 kbits/s). 50% of the slowest 10 download speed countries are in Africa.The median upload speed (303kbis/s) is 1/3 of the median download speed. Currently the African representation by country is only 26 (out of I think 54). It will be interesting to see if this improves as more measurements are made.
...
A comparison of the Ookla upload versus download throughput speeds is seen below on the left. Also the ratio of upload/download is seen in the map to the right above.In all cases the download exceeds the upload speed. The best value for the ratio of up/down is 80% (Azerbaijan). The worst is 9% (Portugal), the median is 34% +- 17%. It is seen that several Eastern European countries such as Russia, Romania, Bulgaria, and Moldova have anomalously high upload throughput speeds. Also several Looking at the map of Ookla servers we believe there are several good servers in Romania, Bulgaria and Moldova able to provide high throughput for clients. Several countries in Western Europe such as Portugal, Belgium, France and Germany have anomalously low upload throughput speeds. This may be the servers, or possibly the broadband is configured very asymmetrically.
To validate whether the order of magnitude of the throughputs measured by Ookla agreed with those derived from the PingER losses and Round Trip Time (RTT) measurements we plotted the Ookla speedtest download measurements versus the Normalized Derived throughputs (NDT) from Pinger. It is seen that the order of magnitudes agree and there is a good positive correlation (R2 ~ 0.53). To see how well the Ookla speedtest download results agreed with those of the ZDnet's Australia Speedtest (though the name Speedtest appears in both they are not related) we also plotted the two speedtests against one another. As a comparison we also show the ZDnet Speedtest results vs. the PingER NDT.
Ookla Upload vs download speed | PingER NDT vs. Ookla speedtest transfer rate | ZDnet Speedtest vs Ookla Speedtest | ZDnet speedtest vs PingER NDT |
|---|---|---|---|
| | | |
The figure below shows bar charts of the ratio of upload/download together with the download speeds.
