Pakistani Case Study 2010-2011

Deployment

Monitoring Hosts

In 2010, following a series of workshops and site visits, the team at NUST and SLAC worked with Pakistan’s Education and Research Network (PERN) and Pakistani Universities to put together an end-to-end (E2E) network monitoring infrastructure for PERN connected higher education sites. By the end of 2010 they had have installed the PingER monitoring tools and started gathering data at 18 sites in Pakistan. This includes 4 sites (NUST, COMSATS, PERN and NCP/Quaid-i-Azam) which have been in place for a longer time. In addition they are working on a further 8 monitoring sites. Over 2010 the number of monitoring host – remote host pairs (both in Pakistan) has increased from about 30 to over 220.

The figure below shows the main regions where the hosts are located and how the regions are connected by PERN:

The difficulty of the installations has varied host to host. The technical installation has been simple and not resulted in delays. The delays between deciding to install a monitoring host and gathering measurements from the host have mostly been due to: machine availability; getting administrative approval within university; and delays in making the DNS record entry. The table below shows the history of the installations:

In addition work is in progress to install: BUITMS in Quetta, Bolochistan; Sind Agriculture University Tandojam in Karachi, Sind; NCEMB University in Punjab, Lahore; Virtual University Lahore, Ounjab; Air University in Islamabad, Punjab, Kinnaired College for Women in Lahore, Punjab.

The growth in the number of monitors and host pairs being monitored over 2010, is seen below (spreadsheet ):

Map of sites

The locations of the Pakistani monitoring (red) and remote(red and blue) hosts are seen in the maps below.

It is interesting to see (right hand map above) the large differences in minimum  RTT between PERN and say NCP (at Quaid i Azam university in N.E. Islamabad) of < 10msec (blue line or more exactly 1.3msec) and that between PERN and NUST in the S.W. corner of between 40-80msec (red line or more exactly 44msec.). Presumably this is due to the routing of PERN connections in Islamabad region.

Also interesting to see (middle map above) is that the connection from NUST to the DSL site (blue line) is much less than that to the physically closer NUST host. This host (lo-0-gw.dsl.net.pk(203.82.63.254) is connected via a Micronet broadband connection rather than most of the other hosts which are connected via Nayatel to the PERN backbone. Nayatel in turn owns Micronet. The route from SEECS to the DSL host goes via Nayatel as far as we can trace it. Thus the link from SEECS to PERN which goes via Nayatael may go to a Nayatel Point of Presence at the same location as the DSL host and then to the PERN metroplitan ring. This may be the cause for the lower minimum RTT to the DSL host.

Archive, analysis site

IN 2010, a second instance of the SLAC archive-analysis site was set up at NUST. This provides backup for data and access, and improved performance for Pakistani users. The NUST hosts are are connected (see the traceroute) to PERN via Nayatel.

Measurements

Using PingER, the monitoring hosts ping each remote host with 10 pings every 30 minutes. From this data we are able to measure minimum and average Round Trip Times (RTT), jitter, loss, unreachabilty (all 10 pings fail) and derive throughput and Mean Opinion Score (MOS). The data is gathered from the monitoring sites on a daily basis by the archiving  sites at NUST, SLAC and FNAL.

Results

Unreachability

A host is considered unreachable if none of the pings sent to it are responded to.  To illustrate this we chose a reliable host at SLAC  (pinger.slac.stanford.edu) and analyzed the unreachability of Pakistani hosts seen from SLAC.

It is seen that several hosts exhibit high unreachability. The reasons behind the high unreliability are usually site specific and vary from lack of reliable power and a source of backup power, floods, lack of access to the site when there are problems that require physical access, lack of expertise, and lack of interest from a site.

RTT and Losses for 2010

The minimum RTT to Peshawar and to Islamabad (left hand graph) appears to have reduced dramatically after April 2010. This is partially due to bringing on new hosts that have lower RTT between them.  In April  there was a factor of 2 increase in the number of host pairs (this is seen in the middle and right hand graphs).

If we select the same host-pairs in both say Nov 2010 and April 2010 then the improvement ((ipdv(Apr)-ipdv(Nnov))/ipdv(Apr) in IPDV is about 47%. Thus things have improved with lower IPDVs for the selected host pairs, or in other words the improvement is not just that more recntly added hosts had lower IPDVs.

Throughput

We derive the throughput from the loss and RTT measurements as throughput = 1460*8[bits]/(RTT[msec]*sqrt(loss)) kbits/s. 

 The derived throughput seen  from SLAC in the graph on the left, has increased by roughly a factor of 2 in 5 years. Within Pakistan (graph on the right) the throughput to Quetta is the poorest, followed by Karachi. Since mosts 

Mean Opinion Score (MOS)

The telecommunications industry uses the Mean Opinion Score (MOS) as a voice quality metric. The values of the MOS are: 1= bad; 2=poor; 3=fair; 4=good; 5=excellent. A typical range for Voice over IP is 3.5 to 4.2 (see VoIPtroubleshooter.com). In reality, even a perfect connection is impacted by the compression algorithms of the codec, so the highest score most codecs can achieve is in the 4.2 to 4.4 range. Using the RTT, loss and jitter we derive the MOS.

 It is apparent that the MOS is very variable, and according to the middle graph above appears to be decreasing (getting worse) in time (see left hand and middle graphs). Some of this decrease is due to bringing on new hosts that have poorer MOS performance. If we fix on jusr aggregating the performance for hosts pairs that have been monitored for the whole period we get the graph on the right. This set of hosts consists of: PK.NEDUET.EDU.N1, PK.COMSATS.EDU.N2, PK.NCP.EDU.N3, PK.NIIT.EDU.N2, PK.NIIT.EDU.N7, PK.AUP.EDU.N2, PK.PERN.EDU.N1, PK.UET.EDU.N2 and PK.LSE.EDU.N3. In any case the MOS is well above the threshold of 3.5 mentioned above, so VoIP calls within Pakistan between these hosts should be successful.

Alpha

The speed of light in fibre is roughly 0.66*c (where c is the speed of light in vacuum). Using 300,000km/s as c this yields Round Trip Distance = RTD[km]=100[km/msec]*minimum_RTT[msec] as a way to derive the distance between the two hosts making the minimum RTT measurement. This assume the minimum RTT is only affected by the transmission of light in the fibre (i.e. no delays due to network devices such as routers) and that the fibre route is direct (a great circle route) between the two hosts. The use of minimum RTT is meant to eliminate most network device delays for reasonable fast circuits (e.g. at 100Mbits/s assuming no queuing the router delays is ~ 0.12msec). To accomodate these extra delays one introduces a function alpha, so that RTD[km]=alpha*100[km/msec]*minimum_RTT[msec]. Large values of alpha close to one indicate a direct path, and small values usually indicate a very indirect path. This assumes no queuing and minimal network device delays. The chart below shows the alpha values between regions in Pakistan. It is based on the minimum RTTs seen between Dec 2009 and Nov 2010.

It is seen that the links between Karachi and Lahore, Karachi and Islamabad, and Karachi and Peshawar are very direct (values of alpha close to one) and are also very consistent (low values of the standard deviations).  Islamabad and Quetta apparently are connected very indirectly (low value of alpha). Looking at the map at the top this makes sense since the route goes via Karachi in the South and then back northwards to Quetta. The links between Islamabad and Lahore, Islamabad and Peshawar and Lahore and Peshawar all have lower vales of alpha and thus appear to be more indirect and have higher variability. A common element in the links between these three regions is that they all pass through Islamabad (see PERN backbone map at top).