Executive Overview

A total of six new POP nodes are added in Lahore, Karachi and Islamabad, two each. This helped in better POP to POP intraregional analysis and gave more reliability to measurements. Some POP nodes have unreachability issues for example ISL QAU, LHR GCU and KHI CPSP. Peshawar Region has highest RTT when accessed from any of the other regions.  Also, Peshawar to Islamabad RTT is higher than Peshawar to Karachi RTT. A clear difference of RTT in forward and reverse direction is seen for Intraregional POP in ISL and Lahore. Multan and Faisalabad POP have RTT from Islamabad. Islamabad to Karachi RTT is better than Islamabad to Lahore and Peshawar. Islamabad to Jamshoro POP has high RTT and IPDV each month. NEDUET and GIKI were unreachable all the time.

POP to POP Analysis

We study the performance evaluation in terms of link stability through the data collected by various monitoring sites running through the remote nodes that are deployed at PERN-POPs. The link stability is examined for by means of different performance evaluation metrics that include the throughput, the average and minimum Round Trip Time (RTT), the nodes unreachability and the corresponding packet loss observed. This study includes the performance evaluation between POP to POP nodes and between POP to Non-POP nodes, for the months June-November 2011. Some of the unpredictable situation in the network is supported by use of outliers presented at the end.

The POP to POP node pairs increased from September to October as we added 2 new POP nodes in each of Islamabad, Karachi and Lahore region. Figure 1 shows the increase in POP to POP node pairs. Throughput of POP to POP also increased over six months. Figure 2 shows the increase in POP to POP throughput from June to November. The black line shows the exponential increase trend line. It is observed that new POP nodes were added in three regions which already had one POP node each; resulting in high POP to POP throughput when the Node pairs are within the same region. As a result POP to POP throughput increased as shown in Figure 2.

Inter Region POP to POP Analysis

Overall POP to POP performance was affected by the RTTs. We analyze the average Round Trip Time (RTT) where we take the PERN node deployed at Fatima Jinnah Women University (FJWU) Islamabad as the monitoring node and all the other POP nodes as remote nodes that are being observed. Figure 1 shows the average RTT observed during the months September 2011 to November 2011. It can be observed that the tendency of the RTTs for various months intended for various cities remain identical with a very small variations. As an example, the average RTT observed form FJWU to Karachi HQ (green line with triangles) for the months September-November is 26.59ms, 25.22ms, and 25.14ms. The same tendency is also followed by the other remote sites. Giving a close look to RTTs, it is observed that:

• ISL HQ to Jamshoro Average RTT increased from 45ms in Sep to 91 ms in November. However, its min RTT reduced from 36.3ms in August to 26.8ms in September and remained almost same for the coming months. This increased Avg RTT resulted in reduced throughput.
• RTT for Karachi POPs is very stable throughout the analyzed months.
• The average RTT from FJWU to Quaid-e-Azam University, Islamabad (Islamabad-QAU) is very small (with an average of 0.91ms) compared to the other remote sites.
•  A significant difference in average and Min RTTs is seen for KHI POP, Multan POP and Jamshoro POP. This can be seen in figure 3.

Packet loss and throughput from FJWU POP to other POPs is shown in the tables and average and Minimum RTT is shown in the graphs to identify the nodes with high RTTs.

Intra Regional POP to POP Analysis

Islamabad, Lahore and Karachi regions have three POP nodes each. Intraregional analysis is therefore carried out on these regions. In the tables below, Intra-regional POP performances are evaluated. Each metirc is compared with itself throughout the table (within its column) and colored based on the comparative value. Green shows good values while red shows bad values. It is seen that

• ISL HQ to FJWU unreachability is 12.09% in October however, ISL FJWU to ISL HQ unreachability in October is 0 %.  Similarly, it is seen that FJWU was unable to reach QAU in september for 74% of time however, QAU POP wasunable to reach FJWU 3.7% of time.
• HEC NOC to FJWU and QAU has different RTTs in forward and reverse direction.
• Karachi POP to POP RTT is same in forward and reverse directions however we see a difference in average RTT in Islamabad and Lahore region.

POP to NON POP Analysis

An aerial view of Average RTT of Pakistan shows that the RTTs for Peshawar region are high as compared to other regions.  Multan and Faisalabad POP nodes also show high RTT. Figure 3 shows the average RTTs from SEECS to other nodes in Pakistan with colors of line showing the average RTT for that link.

Figure 3: SEECS to Pakistan Average RTT

Mean Opinion Score (MOS ) is used as the measurement for VoIP. It is a good measure to know which links are good and which ones are bad. Within Pakistan MOS value for all of the links is above 4.1 (which is excellent for VOIP communication). In figure 4 it is seen that MOS (red markers) were not affected much by addition of new nodes (green line shows node pairs). MOS shows that, Pakistan to Pakistan VOIP communication should not be a problem.

Figure 4: Pakistan to Pakistan MOS with Node Pairs

POP to Non-POP Inter-region Analysis

We now examine the POP to Non-POP region performance in order to understand the inter region PERN performance, we chose one reliable node (least unreachable, generally POP node) in a region and try to see how non-POP nodes in other regions look from this node. We have divided all Pakistani nodes in 5 regions, Lahore, Islamabad, Karachi, Peshawar and Quetta. Following are the interesting findings:

1. UETTAXILA performs consistently bad in Islamabad region followed by HEC POP Islamabad node.
2. NEDUET remained unreachable in Sep, Oct and November.
3. IPDV from ISL to Jamshoro is very high (75ms) which results in bad throughput.
4. ISRA University improved its RTT from 65 ms in Sept to 31ms in November.
5. Kinnaird College was unreachable more than 70 % of times with high IPDV. On Nov 20^th there was a spike of Average RTT of 8029.49ms which increased the average RTT to 431ms for the month.
6. Lahore GCU POP node was also unreachable 45.46% and 32.33% in Oct and Nov respectively.
7. In Peshawar Region USTB, Kohat and CAE had high RTT. Kohat had high IPDV of 15ms every month. GIKI was unreachable all the times in Sept, Oct and Nov.
8. Karachi to Karachi and Peshawar to Peshawar average RTT is high (18ms), while for the rest of the regions it is below 1ms.
9. Peshawar region showed worst performance when accessed from any of the regions, including itself.
10. Quetta to Quetta links show low RTT.
11. Peshawar to Islamabad and to Lahore RTT is higher than Peshawar to Karachi and to Quetta.
12. Multan POP, Faisalabad POP and QUEST (Sindh) showed high RTTs.

NOTE: In the tables below, the values are colored while compared to the values of all the regions for the same metric. Color variation is from Red being bad values to Green being good values.

Outliers

Some critical observations that led to finding the outliers are:

• Min RTT from Lahore POP to Islamabad non-POP nodes is low on alternative months.
• Multan to Peshawar RTT is lesser than all other regions. Again UETTAXILA has low min RTT which is even lesser than the Peshawar region.
• PWR POP to SAU min RTT reduced from 41.5 ms in July to 31ms in Nov. Again UETTAXILA has low min rtt of 5.9 ms
• Jamshoro POP to ISL min rtt reduced from 35-40 ms till august to 25-28 ms from Sept onwards. Same goes to Karachi, Peshawar and Quetta. UETTaxila min rtt is same as for other ISL nodes

               i.      ISL POP To Jamshoro POP

Looking at traceroutes from ISl HQ to Jamshoro POP a change in routes is observed.

Traceroute on 25th September:

Executing exec(traceroute, -m 30 -q 3 -f 3, 121.52.154.2, 140)
traceroute to 121.52.154.2 (121.52.154.2), 30 hops max, 140 byte packets
 4  172.31.250.57 (172.31.250.57)  28.641 ms  28.594 ms  30.510 ms
 5  khi77.pie.net.pk (202.125.134.241)  28.016 ms  28.001 ms  28.431 ms
 6  rwp44.pie.net.pk (221.120.251.21)  30.436 ms  30.423 ms rwp44.pie.net.pk (221.120.251.169)  28.412 ms
 7  rwp44.pie.net.pk (221.120.251.58)  31.367 ms  30.371 ms  31.333 ms
 8  khi77.pie.net.pk (221.120.205.70)  31.358 ms  32.286 ms  32.300 ms
 9   (121.52.154.2)  35.265 ms  40.389 ms  37.921 ms

Traceroute on 25th October:

Executing exec(traceroute, -m 30 -q 3 -f 3, 121.52.154.2, 140)
traceroute to 121.52.154.2 (121.52.154.2), 30 hops max, 140 byte packets
 4  172.31.250.57 (172.31.250.57)  25.531 ms  26.403 ms  26.387 ms
 5  tw129-static237.tw1.com (119.63.129.237)  29.595 ms  29.742 ms  30.830 ms
 6  tw128-static161.tw1.com (119.63.128.161)  23.843 ms  24.579 ms  24.558 ms
 7  tw255-static10.tw1.com (110.93.255.10)  27.672 ms  27.656 ms  27.639 ms
 8   (212.73.253.73)  180.369 ms  180.354 ms  180.338 ms
 9  ae-5-6.bar2.Marseille1.Level3.net (4.69.151.13)  212.307 ms ae-4-5.bar1.Marseille1.Level3.net (4.69.151.9)  225.650 ms  224.911 ms
10  ae-3-4.edge2.Marseille1.Level3.net (4.69.143.254)  178.239 ms  182.843 ms  185.761 ms
11  213.242.115.2 (213.242.115.2)  136.229 ms  136.184 ms  136.166 ms
12  static-10GE-KHI494-P01-KHI494-SWB.pie.net.pk (202.125.128.157)  137.212 ms  137.221 ms  137.202 ms
13  rwp44.pie.net.pk (221.120.251.58)  137.812 ms  138.460 ms  138.445 ms
14  khi77.pie.net.pk (221.120.205.70)  139.410 ms  140.131 ms  140.116 ms
15  pinger.hyd.usindh.edu.pk (121.52.154.2)  138.823 ms  138.808 ms  138.793 ms
 
The highlighted lines show the changed hops.

 Karachi HQ POP to USINDH

KHI POP to Non-POP performance was consistent for all of the nodes except for SBKWU and USINDH. Looking at traceroutes, it is seen that in November the packets follow the route which takes a long time. However, in August the routes were different resulting in lower min RTT.

On 3rd November is:
Executing exec(traceroute, -m 30 -q 3 -f 3, 121.52.154.50, 140)
traceroute to 121.52.154.50 (121.52.154.50), 30 hops max, 140 byte packets
4  tw112-static205.tw1.com (221.132.112.205)  1.769 ms  2.054 ms  2.286 ms
5  tw255-static6.tw1.com (110.93.255.6)  2.351 ms  2.501 ms  3.032 ms
6   (212.73.253.73)  112.097 ms  112.324 ms  112.386 ms
7  ae-5-6.bar2.Marseille1.Level3.net (4.69.151.13)  112.544 ms ae-4-5.bar1.Marseille1.Level3.net (4.69.151.9)  113.523 ms ae-5-6.bar2.Marseille1.Level3.net (4.69.151.13)  112.764 ms
8  ae-3-4.edge2.Marseille1.Level3.net (4.69.143.254)  113.340 ms  113.081 ms  113.390 ms
Trace

route on 8th August was:

Executing exec(traceroute, -m 30 -q 3 -f 3, 121.52.155.62, 140)
traceroute to 121.52.155.62 (121.52.155.62), 30 hops max, 140 byte packets
 4  rwp44.pie.net.pk (221.120.251.169)  1.036 ms rwp44.pie.net.pk (221.120.251.21)  1.881 ms  2.111 ms
 5  rwp44.pie.net.pk (221.120.251.58)  14.123 ms  14.340 ms  14.583 ms
 6  khi77.pie.net.pk (221.120.205.70)  15.330 ms  15.565 ms  15.797 ms
 7  172.31.253.66 (172.31.253.66)  16.159 ms  16.393 ms  16.456 ms
 8  usindh.edu.pk (121.52.156.130)  16.957 ms  17.190 ms  17.254 ms
 9   (121.52.155.62)  17.667 ms !X  15.497 ms !X  15.596 ms !X

FJWU POP to QAU POP and FSBD POP

The average RTT from FJWU to Quaid-e-Azam University, Islamabad (Islamabad-QAU) is very small (with an average of 0.91ms) compared to FJWU to other POPs. The reason being the number of hops traversed for this particular path is less compared to other remote nodes, which also reduces the round-trip delay. The hops traversed by the trace-route from FJWU to Islamabad-QAU is compared with the trace-route for FJWU to Faisalabad POP

FJWU to QAU on 16^th November   

   
Executing exec(traceroute, -m 30 -q 3 -f 3, 111.68.96.150, 140)
traceroute to 111.68.96.150 (111.68.96.150), 30 hops max, 140 byte packets
 4  111.68.96.150.pern.pk (111.68.96.150)  0.944 ms  0.920 ms  0.909 ms

FJWU to FSBD on 16^th November

Executing exec(traceroute, -m 30 -q 3 -f 3, 121.52.153.242, 140)
traceroute to 121.52.153.242 (121.52.153.242), 30 hops max, 140 byte packets
 4  khi77.pie.net.pk (202.125.134.241)  26.507 ms  26.492 ms  26.482 ms
 5  rwp44.pie.net.pk (221.120.251.169)  26.565 ms rwp44.pie.net.pk (221.120.251.21)  26.617 ms rwp44.pie.net.pk (221.120.251.169)  26.540 ms
 6  rwp44.pie.net.pk (221.120.254.41)  46.577 ms  46.574 ms  46.563 ms
 7  static.lhr63.pie.net.pk (221.120.208.38)  48.778 ms  48.769 ms  48.804 ms
 8  lhr63.pie.net.pk (202.125.147.50)  49.063 ms  49.310 ms  49.381 ms
 9  121.52.153.242.pern.pk (121.52.153.242)  47.842 ms  47.181 ms  47.061 ms