View Source

Time

The proposed date is September 9/19/2019 8:00pm San Francisco time (this is 8:30 am 9/20/2019 India Standard Time).

Invitees

Bebo White, Saqib Ali, Les Cottrell - SLAC

naman madan <naman.madan25@gmail.com, aayush.2896@gmail.com, assabitha@amity.edu, jamesdavid.1997@gmail.com - Amity

Attendees

Amity team below (Left to right: James David, Naman Madan, Ayush Agarwal(Not Visible), Ananthnarayan Rajappa, Vishwani Sati, Aayush Upadhyay, Gurpreet Singh, Sai Sabitha, Aayush Jain)

Internet End-to-End Performance Monitoring > 20190919 Meeting with Amity > Screen Shot 2019-01-02 at 7.19.11 PM.png

Method

We met by Zoom (see https://zoom.us/). Each attendee will need the Zoom app.

Topic: roger cottrell's Zoom Meeting with Amity

Time: Sep 19, 2019 08:00 PM Pacific Time (US and Canada)

Join from PC, Mac, Linux, iOS or Android: https://stanford.zoom.us/j/235375142

Or iPhone one-tap (US Toll): +18333021536,,235375142# or +16507249799,,235375142#

Or Telephone:

Dial: +1 650 724 9799 (US, Canada, Caribbean Toll) or +1 833 302 1536 (US, Canada, Caribbean Toll Free)

Meeting ID: 235 375 142

International numbers available: https://zoom.us/u/abCOWPniNq

Meeting ID: 235 375 142

SIP: 235375142@zoomcrc.com

Agenda

Deploying Android PingER

The Android version of the PingER MA, is described with comments at ePingER on Android Native - Amity project (this a proposal/description from Aayush Jain)

Amity's documents describe a multipurpose, stand-alone device that can be widely distributed, something that we have brainstormed about for a long time.
Bebo mentioned it to Topher (the Principal Investigator (PI) of the RainForest project) and he feels that the Amity PingER/Android app (when completed and vetted by Topher’s team) could easily be installed as a default service on his rainforest monitors (certainly future ones, not devices already in place). Merging the service data that he already collects with that unique to PingER has the potential to lead to some interesting results.
Since the above Topher has been consumed with RainForest projects and will not have time to devote to PingER/Android.
- A question is whether he can provide Android's and is this needed anyway?

The topics are:

How should we proceed?
- Back in January 2019, we (Topher and the PingER team) agreed to request Amity to share the App and instructions with us; we will look at installing on a jailbroken Android phone at the San Francisco end and try it out.
  - Instructions from Aayush to load the application are at ePingER Functional prototype (for Androids only)
  - We also need to finish up the proxy acquisition of data from the Android MAs. A first step is outlined in https://confluence.slac.stanford.edu/display/IEPM/Proxy+support+for+PingER.
- A later step is to look at, evaluate and find resources to implement the ideas expounded in the Future Proposals section of the paper Amity is working on (is there a more recent copy)
In January they could upload the data from the App to an FTP server at Amity.
- Given concerns about security, there was a discussion about replacing with something a bit more modern such as using a centralized cloud so all can access the data. Amity are looking at Firebase:
  - Firebase Application: The need for a proxy server can be completely eliminated by shifting to a cloud-based architecture for managing files. Instead of SLAC pulling in the generated txt files from MAs around the world, the MAs can themselves push these files to a centralized application server hosted on the cloud; which makes it easier for SLAC to access files as per their need. Considering the great degree of integration capabilities Google offers with its products, the flexibility arising from using the Google Cloud Platform would be pronounced. Google’s Firebase is a mobile and web application development platform that provides developers with a variety of tools and services to help develop high-quality apps, that can scale easily as per changing demands, and which delivers 99.99% uptime. The Firebase SDK allows mobile app developers to quickly add critical and reliable functionality to their applications in a short time. The recommended option here would be to leverage the Cloud Storage for Firebase that allows robust uploads and downloads onto the Google Cloud Storage buckets. Apart from the user authentication module that comes bundled with Firebase, developers can also declare file security parameters so as to allow only certain filetypes to be uploaded. Having all user uploaded files in one place will then enable SLAC to access and process files as and when needed. Server-side processing can be done on the Google Cloud Platform as well, thereby eliminating any need for SLAC to maintain its own physical infrastructure. This form of implementation can be highly beneficial in any kind of region of the world, also including remote places, like deep inside tropical rainforests, or places that have recently been hit by a natural calamity. As this implementation model is not dependent on any local measuring agent, the Android mobile apps can directly deliver data to SLAC over any form on internet connection in minimal time with high reliability.
  - At the January meeting we agreed that for the initial stages, FTP is sufficient, the Firebase model allows much greater scalability, especially as the number of Android/ePingER MAs increases.
  - Is there any progress and archiving the data?
- There was also a discussion about utilizing the mobility aspects of Androids in particular with regard to cell towers.
  - This would probably require adding GPS information (Lat/Long) to the actual measurements.
    - This would be easy to implement in the MA.
    - To be backward compatible we would need to make it easy to ignore this extra information.
    - There has been some email discussion on this, is there any progress on adding the Lat/Longs?
  - A concern raised was that cell mobility results in huge changes in RTTs. The current PingER analysis is based on static MAs and is typically based on looking at changes in performance based on more global and temporal influences such as social unrest, earthquakes, tsunamis, economics, education, development etc. Understanding the effects of mobility would require new analyses.
  - The consensus was that, at least in the near term, the current PingER dataflow/analysis would not change. It would/could use the data from Android/ePingER static MAs.
    - This implies the current PingER dataflow/analysis is extended to identify mobile (non-static) MAs, and hence ignore them or treat them differently. Les does not think this is difficult to incorporate once we decide on the format etc.
  - Some thoughts on mobility can be found at PingER and mobility.
Actions:
- Amity will annotate the image of the Amity participants (see image above) with their names. Les sent an email reminder 1/5/2019.
- Amity will send SLAC the final copy of the paper to be presented at a Confluence conference 1/11/2019 entitled. Les sent email reminder 1/5/2019.
  - It would also be helpful if Amity would set up an email list for the Amity team and share the information. A possible name might be pinger-am
- Topher will look at installing the Android App on an Android phone at the San Francisco end and try the Android/ePingER application out.
- Topher asked for which PingER web sites to look at for understanding PingER. Les later sent the following:
  - data aStart at the PingER home page at http://www-iepm.slac.stanford.edu/pinger/. Some important links are:
    Various Reports (under the Reports tab) in particular
    - How Bad is Africa's Internet
    - IEEE Paper 2000
    - Mapping the Digital Divide
    - 2017 Annual Report
    - A Giant Leap in 2016
    Case studies
    For users who want to look at the data see Results/data tab,.
    Introductory stuff
    - Under About tab
    Under Maps
    - Website map (mainly for developers)
- Les will work with Amity to see if they are going to add GPS information to the MA measurements and if so to understand the format and how to ignore them in the PingER dataflow/analyses.
- Les will contact Amity to see how to collect their Android/ePingER data. Les sent email 1/5/2018.

Status of Amity MA

The Amity MA is unreliable so using it for a case study does not appear fruitful. A start on such a paper has been made. Les is working with Amity to try and understand this unreliability (emails 10/6/2018, 12/6/2018, 12/23/2018), they say "There is an internet problem we are taking care of it and will get back soon".The ability to gather data from it is very intermittent, It runs fine for several days, then there is no data for many days.

For example:

From Dec 1 thru Dec 11th we were unable to ping pingeramity.in(202.12.103.71) from December 1 thru December 11 from pinger.slac.stanford.edu. We manually went back and gathered the data.

On December 20th we were unable to ping pingeramity.in from SLAC. Looking in more detail the outage started:

1545191157,12/19/2018 3:45,320.544,320.908,321.415,0
1545192994,12/19/2018 4:16,0,0,0,100 ...
and ended:
1545376588,12/21/2018 7:16,0,0,0,100 1545378366,12/21/2018 7:46,319.922,320.940,321.961,0

Since we gather data around 1am PST or 9am GMT this means we were unable to gather data from Amity on 12/19/2018 and 12/20/2018.

Again we went back and manually collected what we could of the measurements made at Amity MA. Looking at the table of pings from pingeramity.in to www.ncms.ae, we see missing measurements starting just after 3:25am 12/19/2018 until 14:22 12/20/2018.

The main reliability problem was that the IP address was dynamic. It has now been changed to static. There has also been a power problem so they will replace the UPS next week.

Looking at the data(via http://www-wanmon.slac.stanford.edu/cgi-wrap/ping_data_plot.pl?monitor=pinger.slac.stanford.edu&sites=pingeramity.in&begin_day=7&begin_month=11&begin_year=2018&end_day=6&end_month=1&end_year=2019&data) we now see it has been reliable since December 21st 7:46am GMT.