By: Les Cottrell SLAC and Qasim Lone NIIT/SLAC
Country |
Loss before |
Loss after |
Sites affected / total |
---|---|---|---|
Sudan |
< 4.5% |
> 15% |
3/3 |
Bahrein |
0% |
>10% |
2/2 |
UAE |
<4.5% |
>20% |
1/1 |
Jordan |
0% |
>15% |
4/4 |
Oman |
0% |
>15% |
1/1 |
Qatar |
0% |
>4.5% |
1/1 |
Saudi Arabia |
0% |
>4.5% |
2/3 |
India |
0% |
>50% |
2/8 |
Since the effect may have been transitory while the data was re-routed we looked for increases in losses on January 30th. The effect was seen in about 15 countries of the over 150 countries monitored by PingER. This is shown in Table 2 below.
Country |
Loss before |
Loss after |
Sites affected / total |
---|---|---|---|
Egypt |
<1% |
>7.5% |
3/3 |
Sudan |
<5% |
>30% |
3/3 |
Hong Kong |
<0.75% |
>11% |
1/1 |
UAE |
<4% |
>18% |
1/1 |
Bahrein |
<1.5% |
>7% |
2/2 |
Jordan |
<3% |
>7% |
3/4 |
Oman |
<8% |
>13% |
1/1 |
Saudi Arabia |
<1.2% |
>7% |
2/3 |
Syria |
<3% |
>7% |
1/1 |
Indonesia |
< 2% |
>8 |
1/7 |
Thailand |
<0.2% |
>8% |
1/6 |
Bangladesh |
<5% |
> 7% |
2/2 |
India |
<3% |
> 40% |
2/8 |
Sri Lanka |
<3% |
>6% |
2/5 |
Maldives |
<1% |
>12% |
1/3 |
The differences between the daily and hourly tables may reflect the sites/countries abilities to switch to alternate routes.
The Figure below for a host in the United Arab Emirates shows the impact of the cut and the partial recovery 15-20 hours later as the traffic was presumably re-routed with full recovery after about 3 days.
Looking further into data by PingER we can estimate the approximate start time for this event. The exact time is a bit harder to explain but it is generally between 5 AM and 7 AM GMT on January 30th 2008 for most of the countries in table 1 with the only exception of Bahrain where it all started at 9AM. The differences might be due to the measurement sampling rate or the lack of load to cause congestion until people came to work later.
PingER calculates the throughput of different nodes from the Round Trip Time (RTT) and loss using the Mathis formula. It is interesting to see the effect on throughput of the regions suffered from fibre outage. Below is the table which gives us insight on the throughput of countries affected by this outage.
|| Countries || Throughput before (kbits/s)
|| Throughput after (kbits/s)
|| Sites affected / total
||
UAE |
1200 |
21 |
1/1 |
Bahrain |
800 |
23 |
2/2 |
Jordan |
500 |
30 |
3/4 |
Oman |
125 |
18 |
1/1 |
Saudi Arabia |
800 |
30 |
2/3 |
Bangladesh |
400 |
35 |
2/2 |
India |
800 |
38 |
2/8 |
It is interesting that some countries such as Pakistan were mainly unaffected, despite the impact on neighboring countries such as India. This contrasts dramatically to the situation in June - July 2005, when due to a fibre cut of SEAMEWE3 off Karachi, Pakistan lost all terrestrial Internet connectivity which resulted, in many cases, in a complete 12 day outage of services. This is a tribute to the increased redundancy of international fibre connectivity installed for Pakistan in the last few years.
To better illustrate the recovery for the 30 hosts in 16 countries, in the Figure below we show the median RTT and Loss seen from SLAC to these hosts for the last week in Januaryaround this time. The sharp increase in RTT & Loss on January 30th is immediately obvious. It partially recovers on January 31st to about 550 ms and continues to improve for the next few days. The error bars show the variability of the results (as the Inter Quartile Range), which is seen to increase dramatically after January 29th.
It also needs to be stressed that not all hosts in all countries were impacted, e.g. India: 2 of 8; Sri Lanka 3 of 5; Malvinas 3 of 5; Indonesia 1 of 7; Turkey: 1 of 3; Thailand 1 of 6. On the other hand all monitored hosts were impacted in: UAE(1), Bangladesh(2), Bahrain(2), Djibouti(1), Jordan(4), Oman(1), Qatar(1), Saudi Arabia(2). The numbers in parentheses are the number of hosts monitored. The list of countries in this region that have PingER monitored hosts, but none of the monitored hosts were noticeably impacted is also interesting. It includes: Egypt, Ethiopia, Iran, Israel, Lebanon, Pakistan, Palestine, Sudan, Central Asia, the Caucasus and East Asia. Some of these were lucky that they did not make major use of either of the two cables (e.g. Palestine is mainly served by SEAMEWE3), others had totally different routes from SLAC (e.g. via the Pacific for East Asia), others such as Israel had other alternate paths, other used satellite (e.g. Central Asia). We also did not observe complete loss of connectivity, associated with the outage, for any host.
Looking in more detail at the impact hour by hour on January 30th we get the map in the Figure below. It shows the hourly average RTTs (z axis) by time of day (y axis) on January the 30th from SLAC to 30 impacted hosts (x axis) in 17 countries identified by their Top Level Domain (TLD). In this graph the vertical (Average RTT) is chopped off at 2 seconds, though some hosts took up to almost 10 seconds to respond at times. It can be seen that the impact (sudden increase in RTT) is very abrupt. The time of the impact varies by 2-3 hours (between 4am and 7am). Most hosts continued to respond apart from 3 in Sudan and 1 in Bahrain, each of which did not respond for up to an hour. The magnitude of the impact also varies by more than an order of magnitude from country to country. By looking at the data for other days in particular Jan 29th (see the second map below), we verified that the sudden increase was not caused by the normal diurnal variations of people coming to and leaving work etc.