Japanese Earthquake March 11th, 2011

There are some interesting results from the Japanese earthquake of March 11th 2011. According to http://earthquake.usgs.gov/earthquakes/recenteqsww/Quakes/usc0001xgp.phpl  the big (8.9) one was Friday, March 11, 2011 at 05:46:23 UTC.

Overview:

Internet connectivity to the hosts PingER monitors in Japan was maintained. Round Trip Times (RTT) to some hosts increased significantly as seen from SLAC. However as seen from RIKEN in Japan they did not increase. It appears the increase in RTT depends on the route from the monitoring host to Japan. This suggests a possible cable disruption. For more on cables see here.

The Japanese Internet appears to have kept running remarkably well though one of the 6 end hosts monitored did go down for some time after the earthquake. This compares well with the impact of the Chilean earthquake on the hosts monitored in Chile in February 2010 in which both hosts monitored in Chile were not reachable following the earthquake. One was again reachable a few hours later, the other was not reachable until March 3rd 2010. The Japanese phone network did not fare so well as its Internet.

PingER results

All of the 6 hosts that PingER monitors in Japan stayed up  at the time of the earthquake. The hosts monitored are seen in the table below.

However as can be seen from the map below, we were not monitoring a Japanese host near the epicenter. 

On 3/12/2011 we added Tohoku University (www.tohoku.ac.jp), which we were not monitoring previously. It is on the outskirts of Sendai and close to the earthquake. At first  (on 3/12/2011 12:46pm PST) it was not responding. It did respond on 3/14/2011 at 12:45pm PDT.

53cottrell@pinger:~>ping www.tohoku.ac.jp
ping: unknown host www.tohoku.ac.jpExit 2

Also www.jp.kek although responding on 3/10/2011 it was no longer responding at noon 3/11/2011.

60cottrell@pinger:~>ping www.jp.kek
ping: unknown host www.jp.kek
Exit 2
64cottrell@pinger:~>ping 130.87.104.107
PING 130.87.104.107 (130.87.104.107) 56(84) bytes of data.
From 134.79.252.133 icmp_seq=31 Destination Host Unreachable
From 134.79.252.133 icmp_seq=58 Destination Host Unreachable

It was responding again at 1:10pm 3/14/2011 PDT.

54cottrell@pinger:~>ping www.kek.jp
PING wlb00dn1.kek.jp (130.87.104.107) 56(84) bytes of data.
64 bytes from wlb00dn1.kek.jp (130.87.104.107): icmp_seq=0 ttl=237 time=276 ms
64 bytes from wlb00dn1.kek.jp (130.87.104.107): icmp_seq=1 ttl=237 time=276 ms
64 bytes from wlb00dn1.kek.jp (130.87.104.107): icmp_seq=2 ttl=237 time=276 ms

--- wlb00dn1.kek.jp ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2003ms
rtt min/avg/max/mdev = 276.357/276.388/276.437/0.608 ms, pipe 2

Looking at the PingER data it started responding again at Mon, 14 Mar 2011 07:38:32 GMT.

Looking at the plots below we see all hosts except KEK during and after the earthquake maintained reachability. The increase in RTT is also seen to be a spike of several hours. KEK it is seen to lose reachability around 6:00am March 11th and was again reachable on March14th at about 7:00am:

Japanese hosts seen From SLAC

Looking from SLAC there are big increases in the average RTTs and minimum RTTs for some Japanese sites (but not all). The spreadsheet gives more details.

RIKEN seen from the world

Looking at RIKEN (a monitoring mode and so easy to select on and also one of the most affected as seen from SLAC) seen from the world looking at avg RTT and min RTT we see:

• No effect seen from Africa, E. Asia, Europe, L. America, M. East 
• Big effect from N. America (Canada 163ms=>264ms, US 120ms=>280ms) 
• India CDAC Mumbia no effect, Pune 380ms=> 460ms, VSNL Mumbia 360ms=>400ms 
• Sri Lanka no effect 
• Pakistan (we have lots of monitors so should be interesting).  
  • NIIT sees no effect (nb not on PERN) 
  • The PERN (Pakistan Education and Research Network) nodes starting with 111. (apart from UAAR see later, this needs more investigation) see 420ms=>500ms 
  • The PERN nodes starting with 121. See no effect         

Conclusion It is not the site RIKEN that has gone bad, rather it is some of the routes

RIKEN Looking at Japan

Japanese hosts seen from JP.RIKEN.N3 (RIKEN) see no impact on RTT

It looks the problem is in the route to Japan not within Japan itself. I wonder if the undersea earthquake has disrupted some  cables? This appears to be in line with the information from http://www.renesys.com/blog/2011/03/japan-quake.shtml and http://www.networkworld.com/news/2011/031411-quake-damage-to-japan-cables.html?source=NWWNLE_nlt_network_architecture_2011-03-15. A map of the cables from Telegeography is seen below:

Routes

Comparing the routes from SLAC to RIKEN (ping.riken.jp) and from SLAC to NET.APAN.N2 (ns.jp.apan.net) on 3/11/2011, we see RIKEN has more hops and goes Eastwards via the Avenue of the Americas in NY, while APAN goes directly via Sunnyvale near SLAC and then via Pacific Wave directly to Japan. The traceroutes from SLAC to the  University of OSAKA and the University of TOKYO  are similar to traceroute from SLAC to  RIKEN.

However on 3/22/2011 the route to RIKEN went westwards across the Pacific and the RTT was very stable with Min/Avg/Max/Stdev 131.985/132.173/132.607/0.443 ms for 148 pings. The routes to the University of OSAKA and the University of TOKYO were similar.