20190723 PingER Team Meeting

Time & date 

This meeting:  July  23rd 10 pm Pacific time; a day later 10:00 am Pakistan time; 10:30 am India time; 1:00 pm Malaysian & Guangzhou time;  12:00 pm Thailand time; 7:00 am Jordan and 6:00am Turkey.  

Doodle poll invites sent 7/3/2019.

Format

New items and updates are in boldface.

Coordinates of team members:

Mailing list: pinger-my@googlegroups.com for membership see  https://groups.google.com.

Attendees

Invitees:

Wajahat Hussain+ (SEECS), Saqib+ (AUF), Johari+ (UNIMAS);  Adib (Turkey), Dr. Charnsak Srisawatsakul (Ubru), Eyad Ayoubi- (Jordan), Baraa Muslmani+ ( Jordan), Dr. Shadi Jawarneh (Jordan), Bebo White+, Umar Kalim-, Les Cottrell+

•  + Responded via Doodle can attend; - Responded, but unable to attend on this date; ? extra email sent asking if attending 
• Email addresses (all the following are in pinger-my).
  • cottrell@slac.stanford.edu; wajahat.hussain@seecs.edu.pk; saqibutm@outlook.com; johari.abdullah@gmail.com; adibhabbal@karabuk.edu.tr; charnsak.s@ubru.ac.th; eyadayoubi@gmail.com; b.muslmani@yahoo.com; shadi.jawarneh@yahoo.com; bebo@slac.stanford.edu; umar.kalim@gmail.com; xsaifahmadx@gmail.com

Actual Attendees

Bebo, Saqib, Wajahat, Les

Others

Administration

Use of Zoom

Exit plan for PingER - Discussion Topic

Time to start thinking about this. Do we need one?

• Les retired, still active.
• Bebo's SLAC account expires early 2020, he will bow out then, unless we can find a way to renew the account.
  • Les is an emeritus and it appears he cannot sponsor an account based on an earlier attempt to sponsor Faisal Zahid, which we got Umar to sponsor.
  • Umar is no longer at SLAC, but still involved.
• Topher busy getting funding etc.
• In the over two decades since PingER started, the Internet and wide area networking have changed dramatically
  • Speeds, deployment,  reliabililty, number of users and applications have improved exponentially
  • There are lots of other monitoring tools nowadays: perfSONAR, ISP monitoring, Akamai

What does it affect?

• Pakistan monitoring - Wajahat
  • What is the commitment from Pakistan?
• China - monitoring; big data, blockchain - Saqib
  • Is the current historical data sufficient?
• ePingER on Android - Amity, Bebo, Topher
  • This is new, Amity would need to take leadership
  • Topher is very busy with getting funding and other projects
  • Bebo less interested partially due to loss of SLAC account early next year
• Three PingER analysis projects - Amity
  • Title (tentative): Correlation Analysis between network performance and GDP of a country using PingER data. Abstract (tentative):
  • Title (tentative): Comparison of network performance of India and Pakistan using PingER data. Abstract (tentative): 
  • Title (tentative): Data packets loss prediction due to environmental factors. 
  • Is the current historical data sufficient?
• Compare IPv4 with IPv6 and ICMP versus UDP versus TCP pings - Umar
  • Umar in new job may not have resources/time
  • Measurements probably do not rely on PingER project, rather they can be made ad hoc.
• Monthly meetings - Les, Umar
  • Scheduling with doodle - Les
  • Making up agenda and current status - Les
  • Invitation via Zoom - Umar
  • Attending - all

Possible scenarios

• Hard stop
• Soft stop:
  • Continue running monitors, gathering data, archiving and presenting  as long as SLAC provides storage (1TB) and pinger.slac.stanford.edu VM
    • Give up on monitors after email prompt and no resolution
    • Do not add any extra functions
  • Keep historical data publicly available as long as SLAC provides storage (1TB) and pinger.slac.stanford.edu VM
• In either case explore what role the Internet Archive might play in maintaining/storing PingER data.
• Meet with Amity, Topher, Bebo, Les to get their input to
  • Access to PingER historical data
  • their taking leading role
  • ePinger Android futures
• The general opinion was that we move to the Soft stop.

Jordan  (Updated 4/25/2019)

• Baraa is interested in monitoring other Jordanian hosts.  
• 7/3/2019: Requested update and offered to find some Jordanian hosts that are in Jordan and respond to pings. Baraa responded 7/6/2019 "we have nothing new. we tested several sites but it turns out they are using proxies. we are still looking for other sites that are effectively distributed across Jordan."  Les was also unable to find any using Google.
  • Baraa responded 7/7/2019: "We have nothing new. we tested several sites but it turns out they are using proxies.
    we are still looking for other sites that are effectively distributed across Jordan."
  • Also Baraa has just graduated from Princess Sumaya University with a Masters Degree (Major: Enterprise Systems Engineering) and is looking for Universities to continue his education (PhD.)
    • • 7/24/2019 Les sent email to Baraa aksing whether and how might Baraa's future affect the MA in Jordan.
      • 7/24/2019 Baraa responded "I will have to check with my system's administrator regarding that, however I will do my best to keep the MA live." 

Turkey (No update 7/23/2019)

Amity (Updated 7/24/2019)

Amity team photo:

• Amity MA continues to be frequently non pingable from SLAC, and hence no data is gathered. See plot below:
  • I have a crown job that daily checks if the Amity PingER MA is pingable and if so tries to gather the last 60 days of data from Amity.
  • Below is an example of recent Amity data:
    
    
    
• Given Topher does not have the resources to provide assistance, Amity needs to take a leadership role. 
• To facilitate mobile devices they would need to add the lat/long of the Android to each measurement.
  • And SLAC would need to extend PingER to accept the location lat/long data.
• Bebo suggested it would be most useful if Amity were to provide a stand-alone app on the Android app store that people could install at their will. 
  • This would also provide valuable experience to the Amity developer in terms of Android app development.
• Then PingER would need to know of the new MA so it could gather data from it.
• I believe Topher has a stash of Androids he could provide.
• Bebo will contact Amity to discuss futures, also see the recent history at the bottom under Old Information.
  
• Bebo mentioned that Amity work appears to ramp up before the conference in January, maybe they will get more active in the near future.

Adib  (Updated 7/22/2019)

•  Adib submitted to "Socio-economic Development Indices and Their Reflection on Internet Performance in ASEAN Countries" to Journal: Engineering Science and Technology, an International Journal. They have requested a re-submission since it is over the limit of 10 pages. Adib re-submitted 7/22/2019.
  
  

Saqib is undergoing his Mid  Term Probationary Review at the University.

• Bebo is also on the panel but was unclear whether he has all the information in the email package sent to him and is uncertain how to respond.
• Les was also uncertain how to respond, since a trip to Pakistan seemed unlikely and there was no mention of a video conference.
  • Les sent an email  "I am unclear how you wish me to notify you of my review of Dr Saqib Ali’s Self-Assessment report and my experience working with him, since attending a meeting in Pakistan is probably not feasible, and a video/voice conference would be difficult to set up due to the time zone differences. I am therefore attaching a letter of recommendation. Please acknowledge that you have received this and whether any further action is required on my part."  
  • He got the response "I hereby acknowledge with thanks receipt of e-mail with attachment."
  • Les asked "Will you be requiring anything else?  Should I submit the REMUNERATION BILL?"
  • He received  the answer "Yes".
  • Les sent email "Attached is the signed form for remuneration bill for mid-term review of Dr. Saqib Ali. Is anything else required? Please acknowledge you have received this.  Thanks"
  • Les received the answer "OK, thanks."
  • Les sent a copy of his letter of recommendation and the initial request from Faisalabad including all the enclosures to Bebo.

Saqib plans to continue work on PingER monitoring for remote areas, in particular in Pakistan. He will use Android PingERs. This may partially tie into the Amity project.

He will also continue his work on Blockchain and plans to attend a Blockchain workshop in Guangzhou.

The 4th  Call for Proposals for The Asi@Connect Project are now open. Saqib will submit a concept note in WP6 with Wajahat and Les as co-applicants

• From Les: WP6 mentions Pakistan, however, the examples are a bit of a stretch for PingER. Possibly deploying PingER MAs based on the Android in remote regions (NWFP, Gilgit, Balochistan, etc.) The difficulty will be getting collaborators in the remote regions, also probably a lot of travel to shake hands, give encouragement, assist, troubleshoot, etc. A challenge will be convincing the collaborators that they are getting something useful out of this. Maybe need to extend PingER to add some easy to use tools.  Another possibility is reviving the PingER MAs in Pakistan. Again, as Wajahat indicates, it is finding collaborators willing to install and manage the MAs. Some development may be needed to make the installation easier and maybe add remote management. Again an issue may be what do the collaborators stand to gain, and the longer-term continuation of the collaboration as people move on.
  

  WP4 talks about PerfSONAR which is monitoring. Providing training on PingER, the reverse traceroute servers might have some traction.  

   

  For both of these if one could install a mesh of MAs in Pakistan, including the reverse traceroute/ping servers (which are trivial to install on a web server, take minimal maintenance, and are much simpler than an MA) then one could address the problem of locating the lat/long of routers  by trilateration of ping response time. Note though getting the lat longs of end hosts is fairly well addressed, routers are usually identified as being located to the owner (e.g. all ESnet's routers on the US and Europe are defined as being located at Berkeley California (the HQ of ESnet)

Thailand, Charnsak (No update 7/23/2019)

UNIMAS

Need to add Umar Kalim to http://pinger.unimas.my/pinger/contact.php. From the 7/5/2018 meeting: Johari can't ssh into the server so he will go to it on Monday.  He will also upload the new UNIMAS PingER website next week.

Email from Johari 4/21/2019:  I am still very much interested in the project but I have to manage my time better. Will try to join the next meeting and probably get someone from my side to monitor the equipment and make use of the data collected

SLAC (Updated 7/242019) 

Les believes most of the problems caused by the blocking of www-wanmon have been discovered and fixed. www-wanmon has been shut-down.

Les has been working with FNAL to migrate the MA to a new FNAL host, hopefully pretty seamlessly. After the cutover last Friday, there appears to be a problem accessing the data from SLAC due to some permissions need adding at FNAL.

Les has read Umar's documents on PingER Metrics Motion Charts and updated the PingER measurement data, the country populations, the Human Development Index, ICT Development Index and Internet penetration for the Pinger Motion Charts (see https://www-iepm.slac.stanford.edu/pinger/pinger-metrics-motion-chart.html using Firefox and enabling Flash).

Les has arranged to give an invited talk on PingER and monitoring the Internet at the Silicon Valley Samsung HQ 7/17/2019.

Les will be in Beijing China from September 2nd thru 10th,  giving an invited 20 minute talk entitled "First international computer network connection between China and US". at the Symposium on 30 years of BES Physics to be held at the Institute of High Energy Physics, Beijing, P. R. China, from September 5 to 6, 2019. The web page for the symposium is   https://indico.ihep.ac.cn/event/9761/.

Les and Bebo visited the Internet Archive (IA, see https://archive.org/) and initiated a discussion on archiving PingER data (i.e. a mirror for SLAC) at the IA and also giving a talk on PingER at the IA. 

• The IA responded with information on using an IA Python library to archive the data at the IA. Les responded with interest but lack of resources until mid-September at the earliest.  Also maybe after giving a talk at IA (see below) there might be interest in collaborating form some of the attendees.
• The IA also responded pointing to a form to fill out for a request to give a talk at the IA.
  • Les filled out the form a couple of weeks ago but has heard nothing.
  • Les sent an email to the IA to make sure they had received the form and asking if anything else is required.

State of MAs.

Umar compare ICMP and TCP Ping

Umar is looking at the scamper project from CAIDA, see https://www.caida.org/tools/measurement/scamper/. Les will put Umar in contact with the scamper folks (email sent 4/28/2019)

Unfortunately, for the IPv4 vs IPv6 comparison, there is insufficient data (i.e. only about targets)

Next Meeting

Next meeting:  There will be a Doodle poll, for August 12-18  or August 19-22 at 10 pm Pacific time; a day later 10:00 am Pakistan time; 10:30 am India time; 1:00 pm Malaysian & Guangzhou time;  2:00 pm Thailand time; 7:00 am Jordan time. 6:00am Turkey time.  

Old information

Amity moved here 7/24/2019

Android progress:

• Instructions from Aayush to load the application are at ePingER Functional prototype (for Androids only)
  • They sent a new copy of the app 4/6/2019, see ePingER Functional prototype
    • The information has been passed to Topher, who plans to install on an Android. The ball is now in our court
    • There is a discussion between Les and Aayush on how to format the raw data records, in particular with respect to the lat/long. Sent a reminder 4/18/2019.
    • We (SLAC) an email to Aayush asking how we gather the raw data from the cloud. Sent reminder 4/18/2019.
      • Aayish responded 4/18/2019: 
        Yes, we have modified the app to write the semi-colon to the file as a separator.
        However, at this point we suggest you to not make any changes to the pinger project yet. 
        
        
        Let us see and run in this parallel to what is already setup at Amity MA. This will allow us to compare the quality of results by both the systems – one at present which does not utilize the app, and the new one which requires an app for data collection.
        We send the data from the app to the firebase cloud service as JSON objects via a REST API. 
        On the cloud-server side, we intend to build a compute-service that automatically converts the json data to a txt format, for consumption by data researcher team at amity (because they are already used to running analysis on the txt files; but seem to be open to working with JSON data as well). [For the SLAC gatherer we would at least initially prefer the data to be retrieved/available in txt form. This way the changes to the gatherer are minimized and more consistent with current practice.]
        
        
        So once the team finds that the results are significant enough, then we can formally propose to include lat/long data in the Pinger gathering, archiving and analysis.
        For now, it would be best that we do not disturb the current Amity MA, and that the android project runs parallel to this."

• Topher is back, Bebo is contacting him. Bebo's thoughts were that:PingER will need a fixed name for each Android MA.  Since the Android may be mobile the IP address may be dynamic. There is a DNS name to dynamic IP address service that may be useful. Maybe there is another unique fixed identifier in an Android that could be used such as a serial number or SIM ID.
  •  If the Amity app is robust  (i.e. it does not noticeably impact the other services, power, networking, security etc) make it part of the standard Rainforest installation;
  • Topher's package knows the GPS location, so it should be available to PingER for recording.

NUST moved here 7/24/2019

• Wajahat said the letter has been received and a written response is being drafted. Is there an update?
• They (NUST) would also be interested in other collaborations with SLAC
  • Since Les is no longer an employee of SLAC but rather an emeritus this may complicate things.
  • A possibility might be in high-speed data transfer. It would need a champion from NUST, and a plan and funding to provide access to 100Gbps

Wajahat said that contacts at some sites are not very interested in PingER and wonder why there are so many pings. Thus he believes we need new sites representing the regions of Pakistan that have willing collaborators. Thus we should give up on sites such as CAE and ISRA that have not had any data to gather for a year or so.  Les has since disabled these sites.