...
Also interesting to see (middle map above) is that the connection from NUST to the DSL site (blue line) is much less than that to the physically closer NUST host. This host (lo-0-gw.dsl.net.pk(203.82.63.254)) is connected via a Micronet broadband connection rather than most of the other hosts which are connected via Nayatel to the PERN backbone. Nayatel in turn owns Micronet. The route from SEECS to the DSL host goes via Nayatel as far as we can trace it. Thus the link from SEECS to PERN which goes via Nayatael may go to a Nayatel Point of Presence at the same location as the DSL host and then to the PERN metroplitan metropolitan ring. This may be the cause for the lower minimum RTT to the DSL host.
Archive, analysis site
IN 2010, a second instance of the SLAC archive-analysis site was set up at NUST. This provides backup for data and access, and improved performance for Pakistani users. The NUST hosts are are connected (see the traceroute) to PERN via Nayatel.
Measurements
Using PingER, the monitoring hosts ping each remote host with 10 pings every 30 minutes. From this data we are able to measure minimum and average Round Trip Times (RTT), jitter, loss, unreachabilty (all 10 pings fail) and derive throughput and Mean Opinion Score (MOS). The data is gathered from the monitoring sites on a daily basis by the archiving sites at NUST, SLAC and FNAL.
Results
Unreachability
A host is considered unreachable if none of the pings sent to it are responded to. To illustrate this we chose a reliable host at SLAC (pinger.slac.stanford.edu) and analyzed the unreachability of Pakistani hosts seen from SLAC.
| | |
Table of unreachability seen from SLAC to Pakistani hosts in 2010. Higher values (bad) are colored redder. The data is sorted by increasing unreachability in Jan 2011. Spreadsheet | Chart of the unreachability of Pakistani hosts seen from SLAC Dec 2010 and Jan 2011 | Smokeping examples of unreachabillty seen from SLAC for 120 days Oct 2010 - Jan 2011. |
It is seen that several hosts exhibit high unreachability. The reasons behind the high unreliability are usually site specific and vary from lack of reliable power and a source of backup power, floods, lack of access to the site when there are problems that require physical access, lack of expertise, and lack of interest from a site.
RTT and Losses for 2010
| | | |
The average of the minimum RTT measured between regions of Pakistan between Dec-2009 and November 2010. Spreadsheet | Various percentiles for the Inter Packet Delay Variation (IPDV or jitter) between Pakistani monitoring hosts and remote host pairs. The line shows the number of pairs with measurements contributing to the results. Spreadsheet | The blue dots are the median losses seen between all pairs of monitoring and remote hosts for each month. The error bars show the extent of the 25 and 75 percentiles. The red dots are the number of pairs contributing to the packet loss measurements. Spreadsheet |
|
Mr. Muhammad Zeeshan contacted PERN for information regarding exact routing paths. Mr. Jawad from PERN-HEC also pointed out that currently PERN relies on PIE network infrastructure for inter-city communication. However they are laying their own infrastructure (cable, routers) and will be able to use their own dedicated links within 4-7 months. PIE usually forwards traffic through Karachi for now. This means that traffic to Quetta goes via Karachi (as also indicated by our traceroute). It must be noted that PERN is a federated environment connecting all major universities of Pakistan. It must also be noted that the intra-city routes are mostly dedicated. Though all three PingER nodes of NUST-SEECS are connected via Nayatel.
Archive, analysis site
IN 2010, a second instance of the SLAC archive-analysis site was set up at NUST. This provides backup for data and access, and improved performance for Pakistani users. The NUST hosts are are connected (see the traceroute) to PERN via Nayatel.
Measurements
Using PingER, the monitoring hosts ping each remote host with 10 pings every 30 minutes. From this data we are able to measure minimum and average Round Trip Times (RTT), jitter, loss, unreachabilty (all 10 pings fail) and derive throughput and Mean Opinion Score (MOS). The data is gathered from the monitoring sites on a daily basis by the archiving sites at NUST, SLAC and FNAL.
Results
Unreachability
A host is considered unreachable if none of the pings sent to it are responded to. To illustrate this we chose a reliable host at SLAC (pinger.slac.stanford.edu) and analyzed the unreachability of Pakistani hosts seen from SLAC.
| | |
Table of unreachability seen from SLAC to Pakistani hosts in 2010. Higher values (bad) are colored redder. The data is sorted by increasing unreachability in Jan 2011. Spreadsheet | Chart of the unreachability of Pakistani hosts seen from SLAC Dec 2010 and Jan 2011 | Smokeping examples of unreachabillty seen from SLAC for 120 days Oct 2010 - Jan 2011. |
It is seen that several hosts exhibit high unreachability. The reasons behind the high unreliability are usually site specific and vary from lack of reliable power and a source of backup power, floods, lack of access to the site when there are problems that require physical access, lack of expertise, and lack of interest from a site.
RTT and Losses for 2010
| | | |
The average of the minimum RTT measured between regions of Pakistan between Dec-2009 and November 2010. Spreadsheet | Various percentiles for the Inter Packet Delay Variation (IPDV or jitter) between Pakistani monitoring hosts and remote host pairs. The line shows the number of pairs with measurements contributing to the results. Spreadsheet | The blue dots are the median losses seen between all pairs of monitoring and remote hosts for each month. The error bars show the extent of the 25 and 75 percentiles. The red dots are the number of pairs contributing to the packet loss measurements. Spreadsheet |
|
The minimum RTT to Peshawar, to Islamabad and to Quetta (left hand graph) appears to have reduced dramatically after April 2010. This is partially due to bringing on new hosts that have lower RTT between them. In April there was a factor of 2 increase in the number of host pairs (this is seen in the middle and right hand graphs). Also most of the nodes shifted to the PERN network in April and May 2010. PERN network is a major dedicated network, connecting all major universities of Pakistan. The network experiences lower RTT as compared to other private ISP owned networks since usual public traffic is not present on these links. As mentioned earlier PERN is also laying its own infrastructure in order to consequently improve the connectivity, especially to more remote areas such as QuettaThe minimum RTT to Peshawar and to Islamabad (left hand graph) appears to have reduced dramatically after April 2010. This is partially due to bringing on new hosts that have lower RTT between them. In April there was a factor of 2 increase in the number of host pairs (this is seen in the middle and right hand graphs).
If we select the same host-pairs in both say Nov 2010 and April 2010 then the improvement ((ipdv(Apr)-ipdv(Nnov))/ipdv(Apr) in IPDV is about 47%. Thus things have improved with lower IPDVs for the selected host pairs, or in other words the improvement is not just that more recntly added hosts had lower IPDVs.
...
| |
Median derived throughput (blue line) with the 25% and 75% seen from SLAC to hosts in Pakistan from Dec 2003 - Dec 2010. The number of hosts being monitored in Pakistan is seen in the brown line. Spreadsheet | Derved Derived throughput between Pakistani region in 2010. Spreadsheet |
The The derived throughput seen from from SLAC in the graph on the left, has increased by roughly a factor of 2 in 5 years. Within Pakistan (graph on the right) the throughput to Quetta is the poorest, followed by Karachi. Since most monitoring hosts are in the North of Pakistan, in particular in Islamabad, there are mainly long RTTs to Karachi and Quetta and hence low throughput (since throughput goes as 1/RTT).
Mean Opinion Score (MOS)
...
| | |
Median MOS and Inter Quartile Range (IQR) between Pakistani hosts for 2010. | MOS between Pakistani regions | MOS for fixed set of Pakistani hosts by region |
It It is apparent that the MOS is very variable, and according to the middle graph above appears to be decreasing (getting worse) in time (see left hand and middle graphs). Some of this decrease is due to bringing on new hosts that have poorer MOS performance. If we fix on jusr just aggregating the performance for hosts pairs that have been monitored for the whole period we get the graph on the right. This set of hosts consists of: PK.NEDUET.EDU.N1, PK.COMSATS.EDU.N2, PK.NCP.EDU.N3, PK.NIIT.EDU.N2, PK.NIIT.EDU.N7, PK.AUP.EDU.N2, PK.PERN.EDU.N1, PK.UET.EDU.N2 and PK.LSE.EDU.N3. In any case the MOS is well above the threshold of 3.5 mentioned above, so VoIP calls within Pakistan between these hosts should be successful.
...
| Wiki Markup |
|---|
The speed of light in fibre is roughly _0.66*c_ (where c is the speed of light in vacuum). Using 300,000km/s as c this yields Round Trip Distance = _RTD\[km\]=100\[km/msec\]*minimum_RTT\[msec\]_ as a way to derive the distance between the two hosts making the minimum RTT measurement. This assume the minimum RTT is only affected by the transmission of light in the fibre (i.e. no delays due to network devices such as routers) and that the fibre route is direct (a great circle route) between the two hosts. The use of minimum RTT is meant to eliminate most network device delays for reasonable fast circuits (e.g. at 100Mbits/s assuming no queuing the router delays is ~ 0.12msec). To accomodate these extra delays one introduces a function alpha, so that _RTD\[km\]=alpha*100\[km/msec\]*minimum_RTT\[msec\]._ Large values of alpha close to one indicate a direct path, and small values usually indicate a very indirect path. This assumes no queuing and minimal network device delays. The chart below shows the alpha values between regions in Pakistan. It is based on the minimum RTTs seen between Dec 2009 and Nov 2010. |
|
Average Alpha measured between regions of Pakistan with the standard deviations (as error bars) and the number of host pairs contributing to the measurement. Spreadsheet |
indicate a very indirect path. This assumes no queuing and minimal network device delays. The chart below shows the alpha values between regions in Pakistan. It is based on the minimum RTTs seen between Dec 2009 and Nov 2010. |
|
Average Alpha measured between regions of Pakistan with the standard deviations (as error bars) and the number of host pairs contributing to the measurement. Spreadsheet |
It is seen that the links between Karachi and Lahore, Karachi and Islamabad, and Karachi and Peshawar are very direct (values of alpha close to one) and are also very consistent (low values of the standard deviations). Islamabad and Quetta apparently are connected very indirectly (low value of alpha). Looking at the map at the top this makes sense since the route goes via Karachi in the South and then back northwards to Quetta. The links between Islamabad and Lahore, Islamabad and Peshawar and Lahore and Peshawar all have lower vales of alpha and thus appear to be more indirect and have higher variability. A common element in the links between these three regions is that they all pass through Islamabad (see PERN backbone map at top).
Islamabad's intra-city traffic experiences multiple hops (within a few square kms) from source to destination. Moreover outbound Islamabad traffic also experiences a slightly indirect route (multiple hops). Traffic passing between Peshawar and Lahore shows a much direct route. This can be noticed by looking at LHR-ISL, ISL-PSH and LHR-PSH alpha values. Among these three LHR-PSH is the highest (indicative of directness) despite the fact that it passes through Islamabad.It is seen that the links between Karachi and Lahore, Karachi and Islamabad, and Karachi and Peshawar are very direct (values of alpha close to one) and are also very consistent (low values of the standard deviations). Islamabad and Quetta apparently are connected very indirectly (low value of alpha). Looking at the map at the top this makes sense since the route goes via Karachi in the South and then back northwards to Quetta. The links between Islamabad and Lahore, Islamabad and Peshawar and Lahore and Peshawar all have lower vales of alpha and thus appear to be more indirect and have higher variability. A common element in the links between these three regions is that they all pass through Islamabad (see PERN backbone map at top).
Conclusions
- An extensive end-to-end network monitoring infrastructure has been set up for PERN connected universities in Pakistan. Over the last year its has grown from 30 monitoring-remote node pairs to over 500 covering most of the major universities in the main regions of Pakistan.
- At some sites, installation and start up of monitoring hosts was delayed by weak local support.
- There is a great deal of variability in the reliability (unreachability) of hosts. Much of this is due to loss of power. An effort needs to be made to understand and improve power reliability and the provision of backup for several sites.
- Given the measured MOS, VoIP tools such as Skype should work well between PERN connected hosts.
- The low values of alpha lead to the conclusion that there may be a lot of indirect routing in the Islamabad region. Further work with PERN is required to see if this can be remedied.
- The poor throughput performance to Quetta is understandable. More work need to be done to understand why Karachi looks so bad.
...
We acknowledge the patient and persistent efforts by Muhammad Zeeshan and Fahad Gilani of NUST to spearhead the installations of PingER at the various sites. Anjum Naveed and Adnan Kianai Kiani led the efforts at NUST. The PingER data was collected and analyzed by Les Cottrell of SLAC and Amber Zeb and Sadia Rehman of NUST. A few PingER tools and procedures were enhanced at NUST by Zafar Gilani. The PingER map tool was developed by Faisal Zahid while at SLAC and turned out to be extremely effective in drilling down and understanding the connections. The Smokeping tool was developed by Fahad Satti while at SLAC. Umar Kalim of Virginia Tech provided support for this year and has been spearheading the effort for last several years. We also acknowledge the continued encouragement and support from Arshad Ali of NUST.