Introduction
In response to a reported performance problem between SSRL and ANL reported by Brian Kobilka's group, Phil Reese of Stanford campus looked at the perfSONAR Bwctl (iperf) plots shown below.
Stanford to ESnet at Sunnyvale |
Denver/ESnet to Stanford |
Chicago/Esnet to Stanford |
ANL/ESnet to Stanford |
|---|---|---|---|
|
|
|
|
It is seen that as the distance from source to Destination increases, the asymmetry between the two directions also increases. This could be due to loss in one of the paths (the one to Stanford). Any loss can dramatically degrade performance. See for example:
- http://fasterdata.es.net/performance-testing/perfsonar/troubleshooting/packet-loss/
- http://staff.psc.edu/mathis/papers/JTechs200105/mgp00020.html
- The following graph might be useful: it shows the random loss rate affect on tcp flavors across a dummynet on a 250mbps link (this was before gig-e was common place). 162ms. Single tcp flow. The full analysis available at http://www.slac.stanford.edu/~ytl/thesis.pdf (~pp152). warning: long and arduous content!
- Also see the ESnet presentation at TERENA 2013 , in particular slides 22 onwards.
The problem was reported to trouble@es.net. It was entered as trouble ticket ESNET-20130528-009
Seen from ANL
From Phil Reese:
Poking at the other end, I found another perfsonar at ANL collecting throughput traffic back toward the west cost. See anlborder-ps.it.anl.gov and look at the throughput graphs available there.
From those, it seems like there is symmetry to Kansas at least but then it degrades as the route moves further west.
From Stanford, we are primarily an I2 site but it looks like once we get into the CENIC world, there is a blending of routes between I2 and ESnet. Not sure if that is a problem or not.