PingER and COVID-19

Introduction

This addresses the question of whether the PingER data can identify the impact of COVID-19 on the Internet. According to https://thrivenextgen.com/covid-19-impact-on-internet-performance/ Most Internet Service Providers (ISPs) oversubscribe their bandwidth and networks as typical inbound and outbound traffic are bursty and often don’t sustain high levels on a continuous basis. Fortunately, most of the larger National Providers have had sufficient time to prepare for the impact of COVID-19 and plan for the possibility of business workloads shifting to the home. Regardless of this capacity planning, Internet traffic patterns are about to change drastically based on schools across the country opting for virtual learning and business work from home migrations. Also, Time Magazine April 5/ April 15, 2020, reports that "traffic worldwide is up 35%...Demand is highest in the evening in the past two weeks, says networking firm Century Link."

For example, one might expect that as schools shut, people self-isolate, are sent home from, work from home, lockdowns are imposed etc., physical person-to-person communications would migrate to the Internet and will increase the use of the Internet e.g. by virtual learning, streaming. communicating, gathering information and entertaining.  The thought is that this will lead to different Internet patterns and possibly lead to Internet congestion. A study of the impact of COVID-19 on Internet speeds by Ookla  ("TRACKING COVID-19’S IMPACT ON GLOBAL INTERNET PERFORMANCE") is directly relevant, however,  it is hard to identify any dramatic impact on the latencies in the report. There is also COVID-19 impacts on Internet traffic: Seattle, Northern Italy, and South Korea which looks at the impact on Internet traffic volumes. Also companies such as Netflix, Amazon, Apple, Disney, Google have reduced the quality of their videos to help reduce traffic and congestion of the Internet (see https://www.traffic-masters.net/covid19-web-traffic-statistics/).

PingER uses the ping facility to measure the Round Trip Time (RTT) between about 20 monitoring sites and target hosts in over 160 countries (over 10,000 pairs of monitor-remote(target)_sites). Every 30 minutes the monitor sends up to 30 pings at one-second intervals to each target until there are 10 responses, and records the data which are daily gathered, archived, analyzed and presented at the archiving site.

• A timeline for COVID-19 can be found here. Also looking at a map of how the COVID-19 traveled the world or the HealthMap one can see how and when various regions were impacted. 
• For detailed statistics see:
  • Coronavirus Disease (COVID-19) – Statistics and Research
  • COVID-19 Coronavirus - Update
• For the raw data you can get up-to-date raw data on the coronavirus from Johns Hopkins from their GitHub site: https://github.com/CSSEGISandData/COVID-19. They have daily summaries and time series data, both available as CSV files. You just have to browse around a bit to find what you want.
• For scientific modeling of the COVAD-19's impacts see the Imperial College London reports on the Impact of non-pharmaceutical interventions (NPIs) to reduce COVID19 mortality and healthcare demand and Estimating the number of infections and the impact of non-pharmaceutical interventions on COVID-19 in 11 European countries. There are also projections for US hospital resources and deaths.

Note that for  Europe, US and Canada the Christmas/New Year season (end November thru 2nd week in January) may also be expected to change behaviors.

Milestones and Statistics

The milestones, for the virus itself, include 1st identified case, 1st hospitalization, 1st community case, 1st death.

Other milestones are the social impacts that are more likely to impact the internet traffic. These include the social interventions:

• Social distancing encouraged
• Case-based measures
• School closures
• Public events banned
• Lockdown
• Interventions removed

Also, see COVID 19 Internet Usage timeline for how companies have reduced their Internet traffic.

Statistics:

• Number of days to double cases
• Basic reproduction number R0 the average number of secondary infections produced by an infectious case where everyone is susceptible.
  • For R >1 each individual infected by a transmissible disease is expected to infect a number of individuals that increase exponentially. For R< each case transmit the disease to less than one individual and the disease dies out.
• Control reproductive number RC, the value of R in the presence of control measures.
• Incubation period
• Risk of importation

Monitoring

The PingER monitor at SLAC, Stanford University in California currently monitors about 100 hosts in China. Using the pingtable.pl web page we selected daily aggregated data from SLAC to China for the last 365 days and looked at various metrics derived from the RTTs. Since many of the targets are Universities that may have taken lockdown measures and sent many students and staff home, we decided to look at the overall impact on SLAC to China pings. To do this we scroll down the pingtable.pl page to summary tables as shown below.    

We can cut and paste this table into Excel to enable further analysis and visualization. Also the summary data is analyzed on a regular basis and the results made available as CSV files at https://www-iepm.slac.stanford.edu/pinger/prmout/. The file names (e.g.average_rtt-EDU.SLAC.STANFORD.PINGER-continent-allmonthly.csv) provide the metric (average_rtt in example), the Monitoring Agent (EDU-SLAC.STANFORD.PINGER in example), the region (continent or country) and the time window of each data point (or Tick, e.g. monthly (=last 12 months), allmonthly, or allyearly).

China and North America

Looking at the plot of daily average RTTs there is a peak  (going from 145ms to 160ms) in minimum (average RTT) from Jan 10 to Jan 24th 2020, plus another spike on Feb 19.

A similar result is seen for the daily minimum RTTs.

 Currently I do not know why we see these bumps or what they are telling us.

Looking at the Inter Packet Delay Variation (IPDV, ) a measure of the variability between adjacent ping packet RTTs there is some evidence that the variability reduced between 18th Jan and March 10th 2020.  The first person in China to die of the COVID-19 virus was reported on January 10th, and by January 31st there were 9,720 cases in China. Looking in more detail at the Healthmap one sees

Looking in more detail at the Healthmap one sees the dramatic spread of the virus in China between January 13 and January 20, 2020. Thus the drop in IPDV RTTs may indeed be linked to the impact of the growth of COVID-19 infections in China.

January 13, 2020	January 20, 2020

 I can see nothing similar when looking at the US and Canada (N. America) IPDV  data, maybe the effect is later in the evolution of the virus' impact. 

Aggregating the IPDV data monthly for China we see the following where we show the median together with the 25% and 75% going back to September 2017:

Looking at the percent packet loss for China there may be some evidence for a change in the 75%. Thus the impact is seen for a few sites. However, the climb started in August-September 2019 so it is doubtful it is related to COVID-19. This conclusion is confirmed by looking at the monthly aggregates.

For the US and Canada the losses are lower and usually 0% (typically >=90% of the targets show no losses), so instead we show the average maximum losses

PingER monitors two hosts in Hubei: www.hbmy.edu.cn at Hubei University for Nationalities and Hubei University of Automotive Technology and www.huat.edu.cn at the Hubei University of Automotive Technology. The locations of the two Universities is shown below.

The hourly measurements for www.hbmy.edu.cn and www.huat.edu.cn are shown below (one can use the form https://www.slac.stanford.edu/cgi-bin/ping_data_plot.pl to create such a plot).

In general there are marked diurnal changes. There is a slight increase in RTTs following the Chinese New Year January 25th 2020. Then there is a small step reduction in RTTs March 10, 2020, followed by an increase in the packet losses starting March 18th 2020.

PingER average hourly RTT from SLAC to Hubei University for Nationalities	PingER percent hourly packet loss from SLAC to Hubei University for Nationalities

If we look at the daily aggregates for the packet loss per cent, average RTT and Inter Packet Delay Variation www.hbmy.edu.cn and www.huat.edu.cn from 3/23/2019 thru 3/21/2020  and then expand to see the subset from 1/1/2020 thru 3/21/2020 we get the plots below. 

Note that www.

It is clear that there are huge variations in the measurements over the last year. Looking at just 2020 data there does appear to be evidence for an almost factor of 2 change in the IPDVs towards the end of January, in opposite directions for the two hosts. N.b we might expect to see something following the January 23rd lockdown of the Wuhan region. There is a "flutter" in the IPDV around that time, whether it is related to the shutdown is unclear.

Looking at the aggregated monthly IPDVs for the two hosts we see:

Europe
 

Italy jumped from three incidents on Friday, February 21 to 229 by the time of WHO’s February 25 situation report

For Italy, PingER monitors 7 hosts. They are in Bologna, Udine, Rome, Pisa and Frascati. These are not the hardest hit areas (see here).

After removing one (vr1dx.fisica.uniud.it) which stopped responding on February 20, 2020 the IPDV for each of the Italian hosts appears as below:

There appears to be a dramatic drop in the IPDV for the target host speed.unidata.it in Bologna between the end of January and Feb 19th. Italy's first COVID-19 case was announced February 19th, thus it is doubtful this drop has anything to do with COVID-19.

A lockdown began for Northern Italy on March 8th and extended to all Italy on March 9th. A plot of the daily IPDV measured from SLAC to 6 Italian hosts together with some COVID-19 milestones is shown below. The milestones are 17 cases diagnosed Feb 21 (including first local transmissions) and 225(105 new) Feb 25. The progress of COVID-16 in Italy can be seen here. There are 6 Italian hosts being monitored from SLAC the values shown are the 25%, median and 75% of the daily IPDV measured for the 6 hosts for the given days.

More Information.

Timeline of COVID-19

Imperial College reports: 

• Impact of non-pharmaceutical interventions (NPIs) to reduce COVID19 mortality and healthcare demand, 16 March 2020
• Estimating the number of infections and the impact of non-pharmaceutical interventions on COVID-19 in 11 European countries, 30 March 2020

MIDAS Online portal for COVID-19 Modeling Research

Potential Long-term Intervention Strategies for COVID-19 

Probably the best source of PingER data is https://www-iepm.slac.stanford.edu/pinger/prmout/ which is aggregated data for various metrics by region and time window (or tick).

• Excel spreadsheets.

  PingER data related to COVID-19 outbreak	COVID-19 cases and deaths by country
  covid19-pinger.xlsx	Log-Covid.xlsx