...
In addition work is in progress to install: BUITMS in Quetta, Bolochistan; Sindh Agriculture University Tandojam in Karachi, Sindh; NCEMB University in Lahore, Punjab; Virtual University in Lahore, Punjab; Air University in Islamabad, Federal Territory; Kinnaired College for Women in Lahore, Punjab.
The growth in the number of monitors and host pairs being monitored over 2010, is seen below (spreadsheet ):
Map of sites
The locations of the Pakistani monitoring (red) and remote(red and blue) hosts are seen in the PingER maps below.
...
Using PingER, the monitoring hosts ping each remote host with 10 pings every 30 minutes. From this data we are able to measure minimum and average Round Trip Times (RTT), jitter, loss, unreachabilty (all 10 pings fail) and derive throughput and Mean Opinion Score (MOS). The data is gathered from the monitoring sites on a daily basis by the archiving sites at NUST, SLAC and FNAL.
Results
Unreachability
A host is considered unreachable if none of the pings sent to it are responded to. To illustrate this we chose a reliable host at SLAC (pinger.slac.stanford.edu) and analyzed the unreachability of Pakistani hosts seen from SLAC.
| | |
Table of unreachability seen from SLAC to Pakistani hosts in 2010. Higher values (bad) are colored redder. The data is sorted by increasing unreachability in Jan 2011. Spreadsheet | Chart of the unreachability of Pakistani hosts seen from SLAC Dec 2010 and Jan 2011 | Smokeping examples of unreachabillty seen from SLAC for 120 days Oct 2010 - Jan 2011. |
...
The minimum of the RTTs measured between Islamabad PingER hosts for each month for 2010 is seen below
|
Minimum monthly RTT measured between Islamabad region PingER Hosts from December 2009 through January 2011. Large outliers Outliers are colored yellow. Green indicates when there was a change from month to month that persisted. Spreadsheet . |
It is seen that there we were several major changes in minimum RTT between the months of June and December 2010. Most of these resulted in an increase in minimum RTT. Some of this was caused by moving connections for a site from one PERN core router to another (e.g. COMSATS), or to a campus or host physically moving from one site to another (e.g. IIU, SEECS). In general these moves resulted in an increase in minimum RTT.
RTT and Losses for 2010
...
Patterns shown by min RTT
...
There was a big improvement in the min RTT to IIU from all sites between Sep and Oct 2010. The reason is IIU's shift to PERN network. It's IP is 111.68.97.162 (IP addresses starting with 111 and 121 are on PERN network) so this confirms that it is now on PERN network. The reason for higher min RTT was basically due to IIU using the public/service-provider network. A screenshot of pingtable shows an observation of this change.
NEDUET (Karachi) from PERN (Islamabad) and other sites is < 30ms which is less than PERN to NIIT (Islamabad). This is due to the fact that NIIT (Islamabad) is not on PERN network. It uses public network (Nayatel to be precise). Only IPs that start with 111 and 121 (e.g. of the form 111.nnn.nnn.nnn and 121.nnn.nnn.nnn) are on PERN network. Further trends are shown below in the table. Spreadsheet here.
Trends:
- LHR to LHR is good (green).
- PWR/NWFP to LHR is good (green). PWR and NWFP belong to the PWR (Peshawar) region. This traffic goes via ISB region.
- ISB shows miscellaneous patterns and this is due to nodes being on different networks and following different routes:
- Nodes in ISB that are on PERN network show good performance (green). These are: pinger.pern.edu.pk (121.52.147.253), vle.iiu.edu.pk (111.68.97.162), pinger-ncp.ncp.edu.pk (111.68.99.142), www.pieas.edu.pk (111.68.99.199) and pinger.uaar.edu.pk (111.68.99.248).
- Nodes not on PERN network show higher RTT (orange and red): SEECS maggie1, maggie2, monitor nodes and pinger.comsats.edu.pk (203.124.40.43).
- We do not fully understand why pinger.comsats.edu.pk (203.124.40.43) has lower min RTT to pinger.nwfpuet.edu.pk (121.52.148.71) and pinger.pwr.nu.edu.pk (121.52.148.110) than vle.iiu.edu.pk (111.68.97.162) or maggie1.niit.edu.pk (115.186.131.81) since COMSATS and NIIT are on public network and NWFPUET, PWR and IIU are on PERN network. However some explanations that come to mind are:
- COMSATS node experiences higher packet loss for nodes at NIIT (around 6.6%) whereas a low packet loss for NWFPUET and PWR (0.48% and 0.40%). We send 10 packets every 30 minutes so this means it loses about 32 packets every day out of 480 for NIIT and only 2 are lost for IIU, NWFPUET and PWR. This is enough to impact the average minimum RTT for the day and consequently for the entire month. Shown in figure below.
- NIIT node is on public network and also experiences a lot of traffic. Therefore there are higher delays and thus higher min RTTs.
- Why COMSATS has higher min RTT to IIU is an anomaly since it has lower min RTT to other PERN connected nodes such as NWFPUET and PWR.
- NIIT has better access to NWFP than does IIU or PERN. Possible reason could be ping unreachability. Unreachability is higher from IIU (46.25%) and PERN (39.17%) as compared to NIIT (36.39%). Detailed stats here. Throughput is higher from IIU and PERN to NWFP; 5218 kbps and 7048 kbps respectively, whereas throughput from NIIT falls to 2259 kbps. Detailed stats here.
- PWR-PWR and PWR-NWFP is good (green). PWR and NWFP belong to PWR (Peshawar) region.
- TAX to LHR is better than TAX to ISB:
- pinger.uettaxila.edu.pk (121.52.150.164) is on PERN network. Most nodes in Lahore are on PERN network: LSE (111.68.102.40), pinger.lcwu.edu.pk (111.68.103.135), pinger.nca.edu.pk (111.68.102.69), pinger.lhr.nu.edu.pk (111.68.102.125), pinger-itc.pu.edu.pk (111.68.103.29) and pinger.uet.edu.pk (111.68.102.14) but about half of the nodes in Islamabad such as SEECS maggie1, maggie2, monitor nodes and COMSATS are not on PERN network. However this doesn't fully explain why this is so.
- Another inconsistency shown by stats is in inter-city routes. PERN network (shown above in Map of PERN2) is not fully deployed. In fact most of the inter-city routes aren't yet deployed. This is confirmed by the following two traceroutes of February 2, 2011. Traceroutes show that routes use public network (pie.net.pk) of Rawalpindi and Lahore telecom exchanges.
| Code Block | ||||
|---|---|---|---|---|
| ||||
3 rwp44.pie.net.pk (202.125.148.157) 7.978 ms 7.978 ms 7.937 ms
4 lhr63.pie.net.pk (221.120.254.2) 14.655 ms 15.388 ms 15.373 ms
5 rwp44.pie.net.pk (221.120.252.226) 14.641 ms 15.355 ms 15.847 ms
6 lhr63.pie.net.pk (221.120.216.198) 16.697 ms 14.582 ms 17.351 ms
7 172.31.240.34 (172.31.240.34) 14.802 ms 14.559 ms 15.279 ms
8 172.31.252.206 (172.31.252.206) 15.270 ms 14.144 ms 98.436 ms
9 nca.edu.pk (111.68.102.69) 98.421 ms 14.677 ms 14.650 ms
|
| Code Block | ||||
|---|---|---|---|---|
| ||||
3 rwp44.pie.net.pk (202.125.148.157) 12.758 ms 7.817 ms 8.080 ms
4 lhr63.pie.net.pk (221.120.254.2) 14.264 ms 15.546 ms 15.538 ms
5 rwp44.pie.net.pk (221.120.252.226) 14.488 ms 14.991 ms 13.693 ms
6 lhr63.pie.net.pk (221.120.216.198) 15.735 ms 20.109 ms 49.523 ms
7 172.31.240.38 (172.31.240.38) 50.542 ms 50.788 ms 53.875 ms
8 172.31.252.230 (172.31.252.230) 51.799 ms 46.953 ms 42.042 ms
9 lhr-nu.edu.pk (111.68.102.125) 14.395 ms 14.163 ms 14.391 ms
|
RTT and Losses for 2010
| | | |
The average of the minimum RTT measured between regions of Pakistan between Dec-2009 and November 2010. Spreadsheet | Various percentiles | ||
| | | |
The average of the minimum RTT measured between regions of Pakistan between Dec-2009 and November 2010. Spreadsheet | Various percentiles for the Inter Packet Delay Variation (IPDV or jitter) between Pakistani monitoring hosts and remote host pairs. The line shows the number of pairs with measurements contributing to the results. Spreadsheet | The blue dots are the median losses seen between all pairs of monitoring and remote hosts for each month. The error bars show the extent of the 25 and 75 percentiles. The red dots are the number of pairs contributing to the packet loss measurements. Spreadsheet |
|
...
If we select the same host-pairs in both say Nov 2010 and April 2010 then the improvement ((ipdv(Apr)-ipdv(Nnov))/ipdv(Apr) in IPDV is about 47%. Thus things have improved with lower IPDVs for the selected host pairs, or in other words the improvement is not just that more recntly added hosts had lower IPDVs.
Throughput
...
(for more information on how throughput is derived please click here).
We derive the throughput from the loss and RTT measurements as throughput = 1460*8[bitsWiki Markup
| |
Median derived throughput (blue line) with the 25% and 75% seen from SLAC to hosts in Pakistan from Dec 2003 - Dec 2010. The number of hosts being monitored in Pakistan is seen in the brown line. Spreadsheet | Derived throughput between Pakistani region in 2010. Spreadsheet |
The derived throughput seen from SLAC in the graph on the left, has increased by roughly a factor of 2 in 5 years. Within Pakistan (graph on the right) the throughput to Quetta is the poorest, followed by Karachi. Since most monitoring hosts are in the North of Pakistan, in particular in Islamabad, there are mainly long RTTs to Karachi and Quetta and hence low throughput (since throughput goes as 1/RTT).
Mean Opinion Score (MOS)
The telecommunications industry uses the Mean Opinion Score (MOS) as a voice quality metric. The values of the MOS are: 1= bad; 2=poor; 3=fair; 4=good; 5=excellent. A typical range for Voice over IP is 3.5 to 4.2 (see VoIPtroubleshooter.com). In reality, even a perfect connection is impacted by the compression algorithms of the codec, so the highest score most codecs can achieve is in the 4.2 to 4.4 range. Using the RTT, loss and jitter we derive the MOS.
| | |
Median MOS and Inter Quartile Range (IQR) between Pakistani hosts for 2010. Spreadsheet. | MOS between Pakistani regions | MOS for fixed set of Pakistani hosts by region |
It is apparent that the MOS is very variable, and according to the middle graph above appears to be decreasing (getting worse) in time (see left hand and middle graphs). Some of this decrease is due to bringing on new hosts that have poorer MOS performance. If we fix on just aggregating the performance for hosts pairs that have been monitored for the whole period we get the graph on the right. This set of hosts consists of: PK.NEDUET.EDU.N1, PK.COMSATS.EDU.N2, PK.NCP.EDU.N3, PK.NIIT.EDU.N2, PK.NIIT.EDU.N7, PK.AUP.EDU.N2, PK.PERN.EDU.N1, PK.UET.EDU.N2 and PK.LSE.EDU.N3. In any case the MOS is well above the threshold of 3.5 mentioned above, so VoIP calls within Pakistan between these hosts should be successful.
Alpha
Wiki Markup
|
Average Alpha measured between regions of Pakistan with the standard deviations (as error bars) and the number of host pairs contributing to the measurement. Spreadsheet |
...
Islamabad's intra-city traffic experiences multiple hops (within a few square kms) from source to destination. Moreover outbound Islamabad traffic also experiences a slightly indirect route (multiple hops). Traffic passing between Peshawar and Lahore shows a much direct route. This can be noticed by looking at LHR-ISL, ISL-PSH and LHR-PSH alpha values. Among these three LHR-PSH is the highest (indicative of directness) despite the fact that it passes through Islamabad.
UETTaxila case study
UETTaxila showed high RTT and unreachability since the node at UET, Taxila came online. A complete case study on this can be found here.
Conclusions
- An extensive end-to-end network monitoring infrastructure has been set up for PERN connected universities in Pakistan. Over the last year its has grown from 30 monitoring-remote node pairs to over 500 covering most of the major universities in the main regions of Pakistan.
- At some sites, installation and start up of monitoring hosts was delayed by weak local support.
- There is a great deal of variability in the reliability (unreachability) of hosts. Much of this is due to loss of power. An effort needs to be made to understand and improve power reliability and the provision of backup for several sites.
- Given the measured MOS, VoIP tools such as Skype should work well between PERN connected hosts.
- The poor throughput performance to Quetta is understandable. More work need to be done to understand why Karachi looks so bad.
- The low values of alpha lead to the conclusion that there may be a lot of indirect routing in the Islamabad region. Further work with PERN is required to see if this can be remedied.
- The configuration of the PERN network naturally changes with time. For example compare the maps: http://pern.edu.pk/images/lapopsislamabad.jpg and http://pern.edu.pk/images/home/download/hld.png where for example CIIT (COMSATS) is show as connected to the QAU router in one and the HEC router in the other. To keep an independent record of the topology we need to measure traceroutes on a regular basis and PERN needs to provide the addresses and locations of the routers.
...