...
Mailing list: pinger-my@googlegroups.com for membership see https://groups.google.com.
Invitees:
Wajahat Hussain?, Kiran Liaqat- (SEECS), Saqib+ (UAF), Bebo White+, Umar Kalim+, Les Cottrell+, Johari (UNIMAS); Adib (Turkey), Dr. Charnsak Srisawatsakul (Ubru), Eyad Ayoubi (Turkey), Baraa Muslmani ( Jordan), Dr. Shadi Jawarneh (Jordan)
- + Responded via Doodle can attend; - Responded, but unable to attend on this date; ? extra email sent asking if attending
- Email addresses (all the following are in pinger-my). The link for the Doodle poll is https://doodle.com/poll/9wc4hxwakqk74wkh
- cottrell@slac.stanford.edu; wajahat.hussain@seecs.edu.pk; saqibutm@outlook.com; bebo@slac.stanford.edu; umar.kalim@gmail.com; kliaqat.msee17seecs@seecs.edu.pk; johari.abdullah@gmail.com; adibhabbal@karabuk.edu.tr; charnsak.s@ubru.ac.th; eyadayoubi@gmail.com; b.muslmani@yahoo.com; shadi.jawarneh@yahoo.com; xsaifahmadx@gmail.com
...
PingER Project Update using Zoom
IMPORTANT NOTE: The meeting is set up to record automatically. By joining the meeting you are agreeing to be recorded (see details)
Bebo White is inviting you to a scheduled Zoom meeting.
Join Zoom Meeting
https://us02web.zoom.us/j/82571424573?pwd=dTJWSmU2TEdPd056ZGRENUJqMityZz09
Meeting ID: 825 7142 4573
Passcode: 874566
One tap mobile
+16699006833,,82571424573#,,,,,,0#,,874566# US (San Jose)
+13462487799,,82571424573#,,,,,,0#,,874566# US (Houston)
Dial by your location
+1 669 900 6833 US (San Jose)
+1 346 248 7799 US (Houston)
+1 253 215 8782 US (Tacoma)
+1 301 715 8592 US (Germantown)
+1 312 626 6799 US (Chicago)
+1 929 205 6099 US (New York)
Meeting ID: 825 7142 4573
Passcode: 874566
Find your local number: https://us02web.zoom.us/u/kccv8XfFPn
When | Thu Oct 29, 2020 9pm – 10pm Pacific Time - Los Angeles |
Where | https://us02web.zoom.us/j/82571424573?pwd=dTJWSmU2TEdPd056ZGRENUJqMityZz09 (map) |
Calendar | cottrell@slac.stanford.edu |
Who |
How to set up and use Zoom:
To use the software, you would have to download the Zoom client (and installed if prompted). The instructions to do so are listed here: https://zoom.us/download#client_4meeting
The instructions about setting up the software are listed at the URL below. You may want to create an account if you do not have one already. If you are invited to a meeting (i.e., you are not hosting the meeting), you are not required to create an account; you can simply join the meeting using the meeting ID.
We had a great meeting as Bebo put it "It was a good mixture of technical brainstorming, future planning, and personal exchange."
Amity
Amity team photo:
Overview
PingER is still actively monitoring from 15 sites in 10 countries (US, Canada, Pakistan, Malaysia, Germany, Switzerland, China, Taiwan, Brazil, South Africa). There are over 700 monitored hosts in 171 countries including all continents except Antarctica. See https://www-iepm.slac.stanford.edu/pinger/sites-per-country.html. Also, see the state of various problem monitoring hosts below.
Bebo
NUST
4/14/2020 Kiran reports: "due to coronavirus we are not going to university. But I have requested the university admin to cooperate with me and install a teamviewer so that I can remotely access the PC and the monitor and maggie2. If the university does so then I will call you on skype so that we can resolve the issue." Kiran has no time estimate as to when this may happen. Kiran's previous post terminated in Sept 2020.
Kiran's contract has been extended as a lab engineer for one year. She was working on monitor, maggie2, nwfpuet. Les has disabled them all and on 10/28/2020 given Kiran details of when they last worked, whether they ping etc.
pinger.vu.edu.pk was fixed July 19th 2020.
Covid-19 see:
UAF (University of Agriculture, Faisalabad)/GHZU
Saqib plans to continue work on PingER monitoring for remote areas, in particular in Pakistan. He will use Android PingERs. This may partially tie into the Amity project.
He will also continue his work on Blockchain and plans to attend a Blockchain workshop in Guangzhou.
Saqib plans to get 2-3 undergraduate students in a month or so and plans to look at Android PingER. No update 4/16/2020.
8/31/2020 there was an exchange of emails: Saqib wrote: I" am healthy and virus-free, however, things turned quite hectic on my end due to online lectures and assignments. Now things are getting better. I have sent a message to pinger.gcwuf.edu.pk to turn the server live again. Further, I am trying to figure out a student to look at the possible impact of Covid-19 interventions on Internet responsiveness as measured by PingER."
UMAR
SLAC
- Bebo posed the question of whether PingER can identify changes in Internet responsiveness as measured by PingER as a result of the impact of the Covid-19 virus.
- The first attempts looking et PingER statistics and see if there were any noticeable changes in performance attributable to Covid-19 interventions (e.g. shutting down schools, social distancing, gathering cancellations, lockdowns), were not very successful (see PingER and COVID-19). See https://covid19-interventions.com/ for interventions in European, and some Asian countries.
- Another attempt to correlate the impacts might be to use a tool such as Google Motion Charts to look for correlations between say Covid-19 cases (cases, hospital admissions, deaths etc) and PingER metrics as a function of time, by country, continent, plus population or density of population. The idea is that as Covid-19 cases pass say 100, the country will impose interventions such as school closing, social distancing etc. which will change the usage patterns of the Internet which may have noticeable impacts on the PingER measurements.
- Unfortunately Google Motion metrics relies on Flash and is no longer supported .
- A possible alternative is Charta.ca
- I have therefore put together a Perl script (chart-covid.pl) to take daily PingER metrics for the last 120 days together with up-to-date raw data on the coronavirus from the Johns HopkinsUniversity GitHub site: https://github.com/CSSEGISandData/COVID-19, for various countries with their population, area, population density and cast the data in a format suitable for Charta.ca.
- There is a web page I put together on how to use Charta.ca at Using Chart.ca.
- There is an example at https://charteca.blob.core.windows.net/pubs/3Z3qcBXqCT9B3WHz
- Instead of using Covid-19 confirmed cases, it might be good to use the Unacast cell phone location data to get a measure of how people are social distancing. I have sent them two emails but no response.
- Discussion:
- Les pointed out that many of our targets (and of course the SLAC Measurement Agent) for the US (ESnet and Internet2), Europe (Dante, and national education/research networks etc.), Australia (AARNET) and parts of Asia (TEIN) are on well-provisioned, lightly loaded networks so it is the transit networks are not expected to see big changes.
- Umar pointed out that the closing of schools, people staying at home should increase the traffic (e.g. between students and profs teaching remotely, people surfing the web watching videos etc) on CDNs (Content Deliverable Networks), and much of this traffic is on public networks. Thus one might see big changes for traffic transitting public networks.
- Since PingER is not monitoring homes and content deliverers, PingER would not expect to see big changes in responsiveness for most of our targets in the above regions.
- On the other hand, African networks are less well developed and PingER may see more of an effect for such targets.
State of MAs.
...
...
...
...
...
Apart from invitations to present papers at upcoming conferences we have not heard from Amity since Sep 4, 2020 when they forwarded an email originating July 11, 2020
Earlier they had said
"We are still working on the portability of the app from Android to Flutter, which will enable the app to be downloaded in iOS environment along with Android. Currently, our application is supported by Android only. Samaksh is looking forward to this issue. We are also discussing how the location feature will add functionalities to our application, and the PingER project will be more than just static servers and we can better monitor the internet with growing mobile communication. Shivang will mail you the list of all the proposed ideas we have on our mind, on what we can do with our mobile application. We will forward the list to Les Cottrell Sir and Bebo White Sir and get it discussed with them too."
"One more thing I wanted to ask you is, can we have the ideas to be working on, or the subdomains, some particular topic we should pursue? Because there are a lot of facts we collected alone in the Coronavirus topic itself, and we need something more specific to work upon, like monitoring some particular countries, the government transparency on data, or measuring the impact the pandemic has on each country on a global scale. Please guide us into this. It would be a great help."
In response we sent them an analysis of the impact of Covid-19 on PingER measurements. See the full report so far (main focus is on Africa and to a lesser extent South America) is at PingER and Covid-19 by Region. At the time we suggested "A possible project for Amity might be to extend the detailed analysis to South Asia. We would be happy to assist as needed."
They responded:
- People's movement and the wave patterns can give us great insight into how the virus is spreading and its impact. With the advent of festivals in India in coming months, we are afraid we may be nearing a wave. We are trying to find its relation with various other factors, one being internet performance. The government has imposed various lockdowns and unlocking with different rules, we are also going to study its direct impact with COVID-19.
Concerning the Android PingER app, the July 11 2020 email said:
We have successfully implemented the Ping through our mobile application, and the data is saved in the default PingER format, with 2 fields in addition, the latitude and longitude of the place.
An android application was implemented before, but this time we made a Flutter app, which makes it possible to distribute the app to both Android and iOS platforms via a single source code.
Regarding the format of data stored, we can only store it in JSON format, because it is the most convenient and best way to store. However, we made a python script which can parse the JSON data to Plain Text format and give the output.
In addition to functionality of the previous app, we are also storing the data locally on the user's phone, which is plain text format in a file, under the folder //Internal Storage/SLAC_Pinger/<date> <time>.txt
Concerning the Android App, Umar pointed out that we need to understand/specify the end goal of the project. The project can support mobility since it records GPS coordinates. One needs to decide how to utilize the data gathered. It can be useful for evaluating Telco service providers. Unlike today's PingER which has continuous data between monitoring stations and targets, we would expect the mobile data to be much more sparse both in terms of time and monitors (cell-phones). Further, it is community-driven rather than centrally. It would appear to be similar to Thousand Eyes or Speedtest app that enables cell phone users to make on-demand measurements of throughput from their phones to targets around the world. The measurements would be archived and can then be summarized and reports presented, e.g. comparing connectivity from area to area on maps and or by a carrier. Telcos are also making such connectivity measurements for their network. An advantage of the current proposal is that it would be across service providers, and also have less network impact than say Speedtest.
We agreed that Les would contact Amity to understand their current status, in particular with respect to the above project (e.g. who is leading the project and involved at the Amity end, the current status) and if relevant suggest setting up a Zoom meeting between the Amity folks and ourselves.
SLAC Overview
PingER is still actively monitoring from 15 sites in 10 countries (US, Canada, Pakistan, Malaysia, Germany, Switzerland, China, Taiwan, Brazil, South Africa). Apart from SLAC, the monitoring sites each monitor 171 Beacons. SLAC monitors over 700 monitored hosts in 171 countries including all continents except Antarctica. See https://www-iepm.slac.stanford.edu/pinger/sites-per-country.html. Also, see the state of various problem monitoring hosts below.
State of MAs.
Host | State | last seen | Status |
---|---|---|---|
daffodilvarsity.edu.bd | No data since Feb 19, 2020. No response to ping. Disabled by Cottrell 10/19/2020. | ||
pingeramity.in | No data since Jan 6, 2020. Cannot ping gives unknown host. Email from Amity Sep 4, 2020: i"Also, for our Amity SLAC server, physical access is not possible and therefore we are unable to configure the scripts manually there and hence it is running offline. It will be set as soon as the college reopens." Disabled by Cottrell 10/19/2020 | ||
pinger.rmutsv.ac.th | Unable to gather data since November 13, 2019. Host is pingable. Sent email to chaisit.c@rmutsv.ac.th 3/26/2020. No response by 4/13/2020. Disabled 10/19/2020 | Nov 13, 2019 | |
pinger.gcwuf.edu.pk | Unable to gather data since 3/31/2020, host is not pingable. Email sent to Saqib 4/13/2020, 10/19/20. Saqib will contact the admins again. | 3/31/2020 | Await saqib |
monitor.seecs.edu.pk | No data since 10/29/2019, host is pingable email to Wajahat 11/7/2019. Reminder to Kiran and Wajahat 2/16/2020. Host not pingable sent email to Kiran and Wajahat 3/3/2020. Email sent to Kiran and Wajahat 4/13/2020. Unknown Host. Kiran's post was to terminate Sep 20. She was working on monitor, maggie2, nwfpuet. No response. Disabled. | 10/27/2019 | |
maggie2.seecs.edu.pk | No data since 11/4/2019, host is pingable email to Wajahat 11/2/2019, 11/20 |
...
/2019. Reminder to Kiran and Wajahat 2/16/2020. Host |
...
no longer pingable, sent email to Kiran and Wajahat 3/3/2020. Email sent to Kiran and Wajahat 4/13/2020. |
...
iran's post was to terminate Sep 20. She was working on monitor, maggie2, nwfpuet. No |
...
progress 10/ |
...
19/2020. It is disabled. | ||
pinger.nwfpuet.edu.pk |
...
Something amiss since April 2019 when it dropped from monitoring over 150 targets down to 4. There is nothing in <BeaconsList>, all the hosts it attempts to monitor are in the locally manually maintained <NodeList>. Does not ping by name. Email sent 11/7/2019, and again 11/20/2019. Reminder to Kiran and Wajahat 2/16 |
...
/ |
...
2020. Email sent to Kiran and Wajahat 4/13/2020. |
...
Kiran's post was to terminate Sep 20. She was working on monitor, maggie2, |
...
nwfpuetDisabled 3/17/2020, we still ping it as a target, |
NUST
4/14/2020 Kiran reports: "due to the coronavirus we are not going to the university. But I have requested the university admin to cooperate with me and install a teamviewer so that I can remotely access the PC and the monitor and maggie2. If the university does so then I will call you on skype so that we can resolve the issue." Kiran has no time estimate as to when this may happen. Kiran's previous post terminated in Sept 2020.
Kiran's contract has been extended as a lab engineer for one year.
...
She was working on monitor, maggie2, nwfpuet
...
. Les has disabled trying to gather data from them all, and on 10/28/2020 given Kiran details of when they last worked, whether the resolve by name, whether they ping etc.
pinger.vu.edu.pk was fixed July 19th 2020
UAF (University of Agriculture, Faisalabad)/GHZU
Saqib plans to continue work on PingER monitoring for remote areas, in particular in Pakistan. He will use Android PingERs. This may partially tie into the Amity project.
8/31/2020 there was an exchange of emails: Saqib wrote: I" am healthy and virus-free, however, things turned quite hectic on my end due to online lectures and assignments. Now things are getting better. I have sent a message to pinger.gcwuf.edu.pk to turn the server live again. Further, I am trying to figure out a student to look at the possible impact of Covid-19 interventions on Internet responsiveness as measured by PingER."
At the meeting, Saqib clarified that the student(s) would be under-graduates, and they would be available in the next 1 to 2 months. This could be an opportunity for collaboration between the UAF team and Amity on the PingER cell phone App.
Saqib mentioned that when he installed PingER in Guangzhou there was pushback from some of the Chinese sites monitored concerning the continuous pinging. We are still monitoring about 100 targets in China.
Covid
Saqib
People do not appear to be cautious and not strictly following guidelines. Yet at the same time the impact in terms of number of confirmed cases for Pakistan compared to India is very different:
While the median age for Pakistan is quite young (late 20's to early 30's) and that the fact the population leans to B+- rather than A+- blood both should help reduce the incidence, this would not appear to be a big difference between India and Pakistan. Saqib mentioned that a difference might be in the way Pakistan imposes a hard 2 week lockdowns (enforced by police with barricades) on small areas (e.g. a street in Islamabad) where there is an outbreak. There may also be some under-reporting.
Saqib also reports that as of the middle of September graduate research students can make short visits to their labs and are resident on campus, while undergraduates are studying remotely.
Umar
Concerning the impact of Covid-19 on work, Umar said that compared to the old regime of being physically at work working, or at home on personal business, compared to today's lack of boundaries where all work is at home, is quite exhausting. For example, the children are not used to parents being at home all the time, communications come during meals, there is a need to indicate when one is not available etc.
Les
Working on analyzing the data from Johns Hopkins University on confirmed cases. and deaths with a library of reports at https://www-iepm.slac.stanford.edu/pinger/excel, including:
- US Politics:
- The impact of Trump's political rallies on spreading Covid-19
- Cases in US states: in favor of, or leaning to, or undecided for Trump or for Biden
- States with Republican governors
- States with Democratic governors
- Cases in: Europe: Central, Northern, Western, Eastern, South-West,
- Cases in: East Asia, Middle East, South Asia, South West Asia
- Cases in: Central America
- I will be adding South America and Africa
- US:
- San Francisco Bay
- Compared to Western Europe
- US most populated states
There are instructions at Covid-19 - Visualizing the data with Excel (https://confluence.slac.stanford.edu/display/IEPM/Covid-19+-+Visualizing+the+data+with+Excel)
Next Meeting
Next meeting: There will be a Doodle poll, for 2020 at 9 pm Pacific time; a day later 9:00 am Pakistan time; 9:30 am India time; 12:00 pm Malaysian & Guangzhou time; 1:00 pm Thailand time; 6:00 am Jordan time. 5:00am Turkey time.
...
Old information
SLAC moved here
- Bebo posed the question of whether PingER can identify changes in Internet responsiveness as measured by PingER as a result of the impact of the Covid-19 virus.
- The first attempts looking et PingER statistics and see if there were any noticeable changes in performance attributable to Covid-19 interventions (e.g. shutting down schools, social distancing, gathering cancellations, lockdowns), were not very successful (see PingER and COVID-19). See https://covid19-interventions.com/ for interventions in European, and some Asian countries.
- Another attempt to correlate the impacts might be to use a tool such as Google Motion Charts to look for correlations between say Covid-19 cases (cases, hospital admissions, deaths etc) and PingER metrics as a function of time, by country, continent, plus population or density of population. The idea is that as Covid-19 cases pass say 100, the country will impose interventions such as school closing, social distancing etc. which will change the usage patterns of the Internet which may have noticeable impacts on the PingER measurements.
- Unfortunately Google Motion metrics relies on Flash and is no longer supported .
- A possible alternative is Charta.ca
- I have therefore put together a Perl script (chart-covid.pl) to take daily PingER metrics for the last 120 days together with up-to-date raw data on the coronavirus from the Johns HopkinsUniversity GitHub site: https://github.com/CSSEGISandData/COVID-19, for various countries with their population, area, population density and cast the data in a format suitable for Charta.ca.
- There is a web page I put together on how to use Charta.ca at Using Chart.ca.
- There is an example at https://charteca.blob.core.windows.net/pubs/3Z3qcBXqCT9B3WHz
- Instead of using Covid-19 confirmed cases, it might be good to use the Unacast cell phone location data to get a measure of how people are social distancing. I have sent them two emails but no response.
- Discussion:
- Les pointed out that many of our targets (and of course the SLAC Measurement Agent) for the US (ESnet and Internet2), Europe (Dante, and national education/research networks etc.), Australia (AARNET) and parts of Asia (TEIN) are on well-provisioned, lightly loaded networks so it is the transit networks are not expected to see big changes.
- Umar pointed out that the closing of schools, people staying at home should increase the traffic (e.g. between students and profs teaching remotely, people surfing the web watching videos etc) on CDNs (Content Deliverable Networks), and much of this traffic is on public networks. Thus one might see big changes for traffic transitting public networks.
- Since PingER is not monitoring homes and content deliverers, PingER would not expect to see big changes in responsiveness for most of our targets in the above regions.
- On the other hand, African networks are less well developed and PingER may see more of an effect for such targets.
Next Meeting
Next meeting: There will be a Doodle poll, for 2020 at 9 pm Pacific time; a day later 9:00 am Pakistan time; 9:30 am India time; 12:00 pm Malaysian & Guangzhou time; 1:00 pm Thailand time; 6:00 am Jordan time. 5:00am Turkey time.
...
NUST moved here 4/14/2020
...