PERN to PERN MOS
We have observed that PERN nodes have gradually improved their Mean Opinion Score (MOS) values over the last year. We derive the MOS values from the PingER average RTT, jitter (measured by the Inter Packet Delay Variability, IPDV) and packet loss. MOS values between 3.5 and 4.4 are acceptable. 4.4 is roughly the maximum achievable with VoIP. Looking at the graph below of MOS values from PERN to PERN hosts, it is obvious that over the last year the MOS values from PERN to PERN nodes have improved from 3.5 to 4.4. In addition the consistency between the PERN pairs as measured using the Inter Quartile Range, IQR (calculated by taking difference of the 75 percentile and 25 percentile), of the MOS values has reduced. This can be seen in the red line on the chart. The decreasing trend of the red indicates that the dispersion between MOS values of the various PERN host pairs is reducing/improving and now the hosts are showing more similar behaviours. We have excluded USTB as an outlier it was not a stable node in October and gave an MOS of 0.05 MOS which is an abnormally low outlier value. The spread sheet is here.
The maps and charts in Fig1 and 2 are obtained from [http://www-wanmon.slac.stanford.edu/wan-mon/viper/pinger-coverage-gmap.html|http://www-wanmon.slac.stanford.edu/wan-mon/viper/pinger-coverage-gmap.html#]. They show the location of PERN hosts in Lahore and the average RTT seen from UET to the other Lahore PERN hosts. Figure 2 is similar and shows the IPDV from UET to the other PERN hosts in Lahore. It is seen that the average RTTs are usually less than 1ms and the IPDVs are < 0.1ms. This indicates excellent RTT performance. (At the start of April 2011, there is a step change in the average RTT from UET to LCWU from 0.4ms to 0.65ms, Anjum is there anexplanation for this?). network is well connected in all cities especially in Islamabad and Lahore see figure below. Within Lahore and Islamabad region,PERN nodes are showing low RTT which is good.
The maps and charts in Figs 3 and 4 are also obtained from [http://www-wanmon.slac.stanford.edu/wan-mon/viper/pinger-coverage-gmap.html|http://www-wanmon.slac.stanford.edu/wan-mon/viper/pinger-coverage-gmap.html#]. They show the location of Islamabad hosts and average RTTs and IPDVs from the Islamabad PERN POP host (pinger.pern.edu.pk alias PK.PERN.EDU.N1) to PERN (IIU, NCP, AIRUNIVERSITY and UAAR) and non PERN (COMSATS and NIIT/SEECS) monitoring hosts. The average RTT from the Islamabad PERN POP node is about 280ms until October 30th 2011, when it drops to about 4ms followed by another drop to 1.5 ms after February 2nd 2011 (Anjum is there an understanding about this?). PERN to PERN nodes RTT is quite small <1ms and IPDV is under 10 which is excellent (see fig 1 to fig 4). For NCP it drops from about 35ms to ~ 1.4ms on December 2, 2011 (Anjum, is there any reason for this?). The AIRUNIVERSITY measurements started on March 26th, s011 and the average RTTs are hovering about 0.45ms. UAAR average RTTs are between 1 and 4 ms.
On the other hand for PERN to non PERN hosts, i.e. SEECS and COMSATS, the average RTT is high >50ms. So PERN is providing more direct connections to the PERN infrastructure for PERN hosts. There are also large noticeable changes in the average RTTs. Also the breaks in the lines indicate considerable unreachability between the pairs of hosts.
|| Fig1: Lahore PERN to PERN node avg RTT
|| Fig2: Lahore Region:PERN to PERN nodes IPDV
|| Fig 3: Islamabad Region: PERN to PERN Impressive Average RTT and NON PERN to PERN node high RTT
|| Fig 4 Islamabad Region: PERN to PERN nodes Impressive IPDV and PERN to Non PERN IPDV
||
| 
| 
| 
| 
|
There are spreadsheets available of the metrics PERN to non-PERN hosts at [IPDV|^pak2pern ipdv.ods] ,[avgRTT|^pak2pern-RTT.ods] ,[Packetloss|^pak2pern pl.ods] and from PERN to PERN hosts for [IPDV|^pern2pern ipdv.ods] ,[avgRTT|^pern2pern rtt.ods] and [packetloss|^pern2pern pl.ods]. From PERN to PERN hosts, all three metrices (packet loss,RTT and IPDV) show slightly better performance than from NON-PERN to PERN host as is shown in Figs 5-7. The MOS is derived from all three of these metrices and their small differences are having a huge impact on MOS. and eventually low MOS is observed from NON PERN nodes to PERN node as clear in fig 8.
|| Fig 5: IPDV: PERN to PERN vs NON PERN to PERN nodes
|| Fig 6. RTT: PERN to PERN vs NON PERN to PERN nodes
|| Fig 7. Packetloss: PERN to PERN vs NON PERN to PERN nodes
|| Fig8. MOS from NON PERN nodes to PERN
||
| 
| 
| 
| 
|
Note:USTB and KOHAT are not considered because they are outliers nodes giving high values to/from other nodes.
SLAC to PERN MOS analysis
From SLAC to PERN nodes, MOS has improved from 3 to ~3.7 with increased consistency which is good. However it seems PERN to PERN nodes are working better than SLAC to PERN. This is as expected since the average RTT affects MOS, and from SLAC to Pakistani hosts, the average RTTs is over 300ms which is much greater than the typical 50 ms say the Islamabad PERN POP monitoring host to other Pakistani hosts. PERN to PERN hosts. This because of high round trip distance from SLAC to Pakistani host compared to that from the PERN Islamabad POP host. The spread sheet is here
Conclusions
- From SLAC to PERN nodes the MOS are in an acceptable range (this is confirmed by the fact we are able to have weekly Skype/VoIP meeting between SLAC in California and SEECS in Islamabad) between 3.5 and 4.
- This is not as good as PERN to PERN hosts for which the MOS is > 4 and improving over the last year.
- PERN to PERN nodes are well connected within cities like Islamabad and Lahore.
- The MOS between PERN nodes is better than between PERN and non PERN nodes such as NIST/SEECS and COMSATS.
- RTT, IPDV and Packetloss from all PERN to PERN nodes is slightly better then from NON PERN to PERN nodes which causing to a lower MOS value.
- There are several major step changes in average RTT between PERN hosts in Islamabad.
- There is still quite a lot of unreachability between the PERN Islamabad POP monitor and other PERN hosts especially in Islamabad.
- We can say that public network between PERN and NON PERN nodes is not working as good as PERN (that is what graphs are depicting).