Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Report

See https://spectrum.ieee.org/tech-talk/telecom/internet/undersea-cable-repairs-south-africa for a very full discussion,  also early reports from https://mybroadband.co.za/news/internet/336416-how-south-africas-internet-survived-cable-breaks.html and the aftermath at https://mybroadband.co.za/news/internet/336426-slow-internet-speeds-in-south-africa-at-least-12-more-days.html.

Both the SAT-3 and WACS undersea cable were damaged Jan 16, 2020. The cable systems are deployed in the Atlantic Ocean and connect various African countries to Europe – including South Afric. According to reports, the issue with the SAT-3 is believed to have been located near Libreville, Gabon. WACs meanwhile is believed to have suffered two faults one of the coast of congo Congo and another near the UK. This affected more than a dozen countries besides South Africa. This left many South Africans with slow Internet connections to international websites. a. That leaves Africa Coast to Coast (ACE) and Equiano providing access to Europe and beyond and SAFE providing access via India and Malaysia and the South Africa Express (SAEx2) which will provide access via India and Malaysia in 2021. 

...

SAT3 (RFS 2002 April)WACS (RFS 2012 May)ACE (RFS 2012 December)Equiano (RFS 2021)

Cables going East to Europe

EASSY RFS 2010 JulySEACOM/Tata TGN-Eurasia RFS 2009 July
Image AddedImage Added

 

Cables going East to India and Malaysia are shown below.

...

RFS = Ready For Service so the current backups to Europe and probably the US are  ACE going West and possibly SAFE SAFECOM both going East.

Result as observed by PingER

...

Tickmin25th%avgmedian75th%90th%95th%maxiqrstd dev# pairs
20Jan01280.444303.117312.614316.141321.182325.130325.531329.47718.06512.63720
20Jan02280.498297.056309.626314.285317.884324.135324.826325.11220.82812.79320
20Jan03280.517303.163312.347316.178321.177324.280324.960329.49918.01412.43320
20Jan04280.461303.154312.365316.158321.146324.649324.978329.48917.99212.45720
20Jan05280.447303.144312.051316.130321.151324.690324.943329.53918.00712.39520
20Jan06280.451296.392310.530315.753320.903324.763324.981329.49624.51113.62820
20Jan07280.468296.436310.834315.841321.240324.988325.506329.45024.80413.77520
20Jan08280.476303.144311.708314.254321.011325.028325.524329.55417.86712.45720
20Jan09280.518303.151311.782314.255319.939323.939325.036328.31416.78811.65820
20Jan10280.465300.798309.671311.542315.925324.952325.496328.31815.12712.84220
20Jan11280.474301.814309.915311.603315.171325.012325.430329.73813.35712.74820
20Jan12280.515296.730311.775315.344322.416324.938329.779330.87725.68614.22820
20Jan13280.429305.275311.683313.730316.397325.007325.491329.70611.12212.04620
20Jan14280.531303.975309.400311.155315.672318.695324.584329.77011.69711.03120
20Jan15296.380305.315313.079312.597318.192324.055325.876336.24812.8779.76020
20Jan16296.726305.008313.498312.889319.100325.828328.229336.27314.09210.34320
20Jan17301.328330.032348.988348.759355.767387.082392.328408.82325.73525.73120
20Jan18301.306330.101345.969348.990355.285366.246385.621389.53825.18421.54620
20Jan19301.268329.970346.692346.227355.276385.389389.286396.48725.30624.19120
20Jan20301.307327.376342.864340.419352.677369.400386.111389.23325.30122.33720
20Jan21301.146327.248346.865342.833352.699396.258412.843415.99025.45130.43520
20Jan22301.350323.368341.581339.179352.644369.499386.202389.18829.27623.34420
20Jan23305.829324.277342.455339.436352.019369.405386.094389.25927.74222.23420
20Jan24305.722323.285341.580338.587352.055366.430386.448389.56228.77022.44620
20Jan25305.680324.671342.747340.071351.452369.217385.966389.60126.78122.06120
20Jan26305.761323.724343.401349.319356.299386.491389.553389.55332.57522.89219
20Jan27305.794321.125341.578338.668352.731369.613386.166389.26131.60623.32120

The targets are

For the 7 targets in South Africa the 25%/median/75% before 2016 and after are ~294/300/306ms and 327/345/356ms.

Looking in more detail at the ping responses each hour we see  a sudden step

Image Removed

and in more detail

Image Removed

The first step occurred between 08:00am and 09:00am Jan 14th, the second step between 07:00am and 08:00am Jan 16th (both times are GMT). I can find no evidence of a failure on January 14th, however it sure looks like there was one. On further inspection, it appears that this step only appears for the TENET target site (see below for the minimum RTTs measured each hour (0 1 ...) from SLAC to the target (Remote-Site)  on Jan 14th, 2020), and thus is not related to the cable problems.

...


For the 7 targets in South Africa the 25%/median/75% before 2016 and after are ~294/300/306ms and 327/345/356ms.

Looking in more detail at the ping responses each hour from SLAC to TENET South Africa we see  a couple of sudden steps.

Image Added

and in more detail

Image Added

The first step occurred between 08:00am and 09:00am Jan 14th, the second step between 07:00am and 08:00am Jan 16th (both times are GMT). I can find no evidence of a failure on January 14th, however it sure looks like there was one. On further inspection, it appears that this step only appears for the TENET target site (see below for the minimum RTTs in msec measured each hour (0 1 ...) from SLAC to the target (Remote-Site)  on Jan 14th, 2020), and thus is not related to the cable problems. This can be seen more clearly in the plot below. The spreadsheets are available here.  

Image AddedImage Added

N.b. two sites (www.bos.gov.ls and www.dut.ac.za) do not appear to have been affected on January 16th or in the daily plots. Maybe they were not using SAT3 or WACS, for example maybe they were using ACE. According to ieee "TENET restored its overseas connections by switching to an alternative 80-gigabit-per-second service on the SEACOM and EASSy cables on Africa's east coast. "The latency ... increased due to this failover, but there are currently no reported issues of this being detrimental to connectivity," a representative of TENET said on 30 January." The impact on the PingER measurements from SLAC was that the minimum RTT went from ~280ms to 355ms.

One can also look at the various statistics for both Average RTT, losses and minimum RTTs, see below (the spreadsheet is seen here). The Average and Losses are heavily influenced by the congestion. The minimum is mainly influenced by the route change or change in the distance of the cable route. Also note that the impact on the minimum RTT is not seen until Jan 17th since some of the PingER January 16 measurements were made before the change of cables used. It is also worth noting as reported in ieee "TENET restored its overseas connections by switching to an alternative 80-gigabit-per-second service on the SEACOM and EASSy cables on Africa's east coast. '"The latency ... increased due to this failover, but there are currently no reported issues of this being detrimental to connectivity," a representative of TENET said on 30 January.'

Image AddedImage AddedImage Added

 


Traceroute

The route measured 19:00 hrs 1/28/2020 PST from SLAC to South Africa goes East across the US via Chicago, Washinton Washington to London and thence S. Africa:

...

AS3671 is ESnet and  AS2018 is TENET

...

ieee Submarine Cable Repairs Underway in South Africa. IEEESpectrum, 3 February 2020, see https://spectrum.ieee.org/tech-talk/telecom/internet/undersea-cable-repairs-south-africa