...
We worked with UNIMAS folks to set uyp a monitoring host at pinger.unimas.my. We noticed that there were severe diurnal variations to most hosts measureed from UNIMAS. We suspected that maybe UNIMAS itself is congested. Since congestion is usually an edge effect since the backbones are well provisioned, We looked at the Round Trip Times (RTTs) from SLAC which is not congested.
this was reported by nara Nara to their network folks
Dear Harun,
We are doing a joint project with Stanford and looks like their testing indicates that our network is congested. Can you check?
I don't want them to move their main link to UM because of this.
...
That sounds very interesting. I assume all traffic will take the new, faster link. Otherwise I hope you will be able to route the pinger traffic via the faster link. Harun says you are shaping the traffic which avoids some of the problems. Looking at the RTT plots (see below) from SLAC to UNIMAS the diurnal changes were much larger (200ms light use periods and multiple seconds heavy use periods) in October. I wonder if the shaping was applied in late October early November, thus accounting from the lower diurnal changes (indicating less congestion) recently. If so identifying the changes in performance maybe a bit more interesting, i.e. improvement with 100Mbps link by adding traffic shaping, and improvement with 500Mbps link with and without shaping. Can this be done, i.e. add and remove the shaping for the PingER host. This will be an interesting application of the use of PingER.
Resolution
It turns out that the traffic shaping was applied at the end of October. This is the likely cause for the reduced diurnal changes and congestion. Also see the time series below where there is seen to be a dramatic improvement in November compared to October.
More details are shown in the time series below. It is seen there are three occurrences of high RTT. they are at about 12:20am 2012-11-14 GMT, and at 18:00 on 2012-11-14 and 2012-11-18. The latter two are at 2:00am Malaysia time (GMT +8). maybe this is due to jobs scheduled (e.g. backup) scheduled at off hours. 