Development Indices and PingER

 Introduction

Classifying countries by their development is difficult. One has to determine what to measure that is related to development and then measure it. There are then costs and practicality concerning: what can be measured, how useful it is, how pervasive it is, how well defined it is, how it changes over time, whether one is measuring the same thing for each country, and the cost of measuring. Various organizations such as the ITU, UNDP, CIA, World Bank etc. have come up with Indices  based on measured items such as life expectancy, GDP, literacy, phone lines, Internet penetration etc. (see below). These measures take time to gather and so are often dated and only available, if at all,  at widely separated intervals.

Another approach is to look at measures of the Internet that can be gathered automatically without running surveys, which in turn can be ambiguous.  Today bandwidth is the life-blood of the information age and the world's economy so one can take the following approach:

"The size of the Internet infrastructure is a good indication of a country's progress towards an information-based economy. ...
But measuring the numbers of users is not easy in developing countries because many people share accounts, use corporate
and academic networks, or visit the rapidly growing number of cyber cafés, telecentres and business services. Furthermore,
simply measuring the number of users does not take into account the extent of use, from those who just write a couple of
emails a week, to people who spend many hours a day on the net browsing, transacting, streaming, or downloading. As a
result, new measures of Internet activity are needed to take these factors into account.
 
One indicator that is becoming increasingly popular is to measure the amount of international Internet bandwidth used by
a country - the 'size of the pipe', most often measured in Kilobits per second (Kbps), or Megabits per second (Mbps).  Most
of the Internet traffic in a developing country is international (75-90%), so the size of its international traffic compared to
population size provides a ready indication of the extent of Internet activity in a country."

Credits:
Research & coordination - Mike Jensen mikej@sn.apc.org
Conceptualisation, management and refinement - Richard Fuchs rfuchs@idrc.ca & Heloise Emdon hemdon@idrc.ca
Design, DTP and Layout: Adam Martin lee@wildcoast.com
Background research and liason: Lee Martin & Rochelle Martin lee@wildcoast.com 

An alternative Internet method is the work of Tom Vest of CAIDA comparing the numbers of Autonomous System Numbers (ASN's) related to Internet deployment using BGP for the measurement data (also see Internet Traffic Exchange: Market Development and Measurement of Growth from the OECD). 

The approach we pursue is to use the end-to-end measurements of the PingER project . This has been gathering end-to-end Internet performance measurements since 1995 and currently measures the end-to-end Internet performance of over 125 countries.  The scatter plot below shows the correlation of PingER loss measurements made for the period Jan-Sep 2007 from SLAC to the world, with the GDP/capita (depicting the productivity of a country) for 2006.

It is seen that there is a moderate to strong correlation R2 ~ 0.58. Similar correlations (R2 ~ 0.52) are seen when one compares PingER derived throughputs vs. GDP/capita.  Stronger correlations are obtained with development indices that are more technology or Internet related. For example if we correlate the PingER performane measurements (jitter, loss throughput) with the GDP/capita and also with one of the more recent and extensive of the indices, namely the ITU's Digital Opportunity Index(DOI) then we get the following R2 values.

The advantage of these automated methods is that they are not subjective, they are available at regular intervals so the derivative with time is relativlely easy to track. The disadvantage is that they are only measure one component of development and care has still to be taken to understand the figures and eliminate false readings. Given the strength of the correlation between the PingER performance measurements and those of GDP/capita and other development indices this is a rationale for pursuing this approach. This document outlines various of the survey based Index classification methods and then goes on to compare them with each other and the PingER measurements.

PingER Measurements 

The PingER project has been described elsewhere. Basically it uses the ubiquitous Internet ICMP echo request/response ping facility to measure or deduce metrics such as  Round Trip Times (RTT), jitter, loss, and reachability from about 40 monitoring hosts to over 600 monitored (remote) hosts in about 130 countries. As part of this project we recognized the need to extend the measurements to more countries. This meant adding some countries. First we identified 84 countries appearing in the various indices that had no hosts monitored by PingER. We addressed these as follows:

• Where no measurement was made since its performance was not significantly diffferent from adjacent countries (this was particularly so in Europe where we added Sweden, Gibralter, Macedonia, Faroe Islands, Austria, Andorra, Bulgaria and Belgium);
• Where previously there had been little interest, in particular Greeenland;
• Mike Jensen gave us a list of hosts in 18 African countries where we previously had no PingER remote hosts. Of these we were able to extract 13 hosts in 8 countries that appeared to be in the country and also responded to pings. These (Guinea-Bissau, Sierra leone, Seychelles, Mauritius, Liberia, Gambia, Swaziland and Djibouti) were added.

Another concern was the fact that the most comprehensive/complete set of measurements was from the SLAC site which measured over 600 hosts. Most other monitors, monitor 200 or fewer hosts. Thus there is a bias that measurements that rely on distance (e.g. RTT) will have better performance if they are close to SLAC. With the current version of PingER we can centrally maintain a list of Beacons (remote hosts that are monitored by all monitor hosts) and automatically update the monitors with the current Beacons on a daily basis.  There are competing concerns regarding increasing the hosts monitored. Each monitor/remote pair adds an extra 100bits/s to the network. Some countries with monitors, such as Palestine have limited bandwidth available. Thus though we need to increase the Beacon list, we need to do this carefully so we do not abuse monitors or remote hosts in countries with poor connectivity. Therefore, for each country with reasonable connectivity one host is selected that is reliable (based on previous PingER measurements), and represents the country. In addition for sites with limited connectivity we will restrivy the ping sizes to 100 Bytes rather than both 100 & 1000 Bytes, i.e. a reduction in traffic by a factor of 10. The idea is to come up with a list of about 120 Beacons covering most countries, thus roughly doubling our current list.  For more on this see PingER Beacons Expansion.

Gross Domestic Product per capita

The gross domestic product or GDP is a way of measuring the size of a region's economy. It is usually normalized by dividing by capita. It is often compared with the purchasing power parity (PPP) of the currency relative to the US$. There are measures from the World Bank and the Central Intelligence Agency among others, we are using https://www.cia.gov/library/publications/the-world-factbook/rankorder/2004rank.html

Human Development Index (HDI)

The UNDP  Human Development Index  report of 2006 was compiled on data from 2004 and covered 175 UN member countries (out of 192). It is a comparative measure of life expectancy, literacy, education, and standards of living for countries worldwide. More specifically:

• A long and healthy life, as measured by life expectancy at birth
• Knowledge, as measured by the adult literacy rate (with two-thirds weight) and the combined primary, secondary and tertiary gross enrollment ratio (with one-third weight)
• A decent standard of living, as measured by GDP per capita (PPP US$).

It is a standard means of measuring well-being, especially child welfare. It is used to distinguish whether the country is a developed, a developing, or an under-developed country, and also to measure the impact of economic policies on quality of life. The index was developed in 1990 by Pakistani economist Mahbub ul Haq.

Digital Access Index (DAI) 

The Digital Access Index (DAI) from the ITU has data from  1995   to 2003. It combines eight variables, covering five areas, to provide an overall country score. The areas are availability of infrastructure, afordability of access, educational level, quality of ICT services, and Internet usage. The results of the Index point to potential stumbling blocks in ICT adoption and can help countries identify their relative strengths and weaknesses.

Network Readiness Index (NRI)

The Network Readiness Index (NRI) was used in the World Economic Forum's Global Information Technology Report 2007-2007. It covers about 120 countries. It rests on three main subindexes:

• the presence of an ICT-conducive environment in a given country by assessing a number of features of the broad business environment, some regulatory aspects, and the soft and hard infrastructure for ICT;
• the level of ICT readiness and propensity of the three main national stakeholders---individuals, the business sector, and the government; and
• the actual use of ICT by the above three stakeholders.

Technology Achievement Index (TAI)

The United Nations Development Programme (UNDP) introduced the Technology Achievement Index (TAI) in 2001 to reflects a country's capacity to participate in the technological innovations of the network age. It contains data from 1995-1000 andf covers 72 countries. The TAI aims to capture how well a country is creating and diffusing technology and building a human skill base. It includes the following dimensions: Creation of technology (e.g. patents, royalty receipts); diffusion of recent innovations (Internet hosts/capita, high & medium tech exports as share of all exports); Diffusion of old innovations (log phones/capita, log of electric consumption/capita); Human skills (mean years of schooling, gross enrollment in tertiary level in science, math & engineering).

Digital Opportunity Index (DOI)

In 2006 the ITU submitted the Digital Opportunity Indexreport for 180 economies worldwide. The Index monitors the mobile communications that promise to bridge the digital divide in many parts of the world, as well as more recent technologies such as broadband and mobile Internet access. Due to it being one with the most recent results and also having a large coverage we tend to prefer this one at the moment.

Correlations Between Indices 

 These are shown below:

Maps

Some maps of the index values are seen below:

PingER metrics

These are described in the Tutorial on Internet Monitoring and PingER at SLAC.

Normalized Derived TCP Throughput 

The normalization is to reduce the impacts of the derived throughput being proportional to 1/RTT. Thus sites close to the measurement/monitor host will have better derived throughputs.  Thus we calculate:

normalized_throughput=throughput * minimum_rtt(for remote_region)/minimum_rtt(monitoring_region)

Comparing this to the GDP/capita we get the scatter plot below. The correlation is seen to be moderate to strong (R2 ~ 0.59). The figure also identifies some of the major outliers. Those  countries below the line are usually well developed but hard to get to countries (such as Finland, Iceland) or wealthy countries that have not full developed their Internet access (e.g. UAE)

Comparisons with loss, jitter and unreachability are shown below together with Jitter vs Loss. I can see why unreachability may not correlate well with loss or jitter, since the unreachability is often an end site/host problem. I am somehwat surprised by the lack of a strong correlation between Jitter and Loss and need to think a bit more deeply about it.  I would have expected a correlation between jitter and loss. Basically jitter is classically caused by router queuing (due to the output link being busy with another packet)  delaying packets. If the output link does not clear then the queue fills with more packets waiting to be sent and newly arriving packets are lost, hence expect correlation. This can be caused by one or more fast links trying to feed a slower or more congested link (e.g. to Developing Country). There can of course be other reasons for loss, such as noise & db loss (especially with wireless and probably satellite circuits) which are not correlated with queuing. I would expect to see the latter causes mainly in places where networking is pooor (e.g. Developing Regions). I see no evidence for the correlation being stronger or weaker at larger values.  At some time I will dig deeper by looking at outliers to see if I can see rhyme or reason.

Comparisons with the DOI are shown below:

Comparisons with GDP/capita are shown below.
 

Comparisons with International bandwidth are shown below: