Time
The proposed date is January 2nd, 7:00-7:45 pm is September 9/19/2019 8:00pm San Francisco time (this is 8:30 am - 9:15 am January 3rd 9/20/2019 India Standard Time).
Invitees
Bebo White, Topher White, Umar KalimSaqib Ali, Les Cottrell - SLAC
naman madan <naman.madan25@gmail.com, aayush.2896@gmail.com, assabitha@amity.edu, jamesdavid.1997@gmail.com - Amity
Attendees
...
Amity team below (Left to right: James David, Naman Madan, Ayush Agarwal(Not Visible), Ananthnarayan Rajappa, Vishwani Sati, Aayush Upadhyay, Gurpreet Singh, Sai Sabitha, Aayush Jain)
Bebo captured the above image of the Amity team. He shared it with the Amity team who will annotate the photo with the Amity team members.
Method
We met by Zoom (see https://zoom.us/). Each attendee will need the Zoom app.
Topic: roger cottrell's Zoom Meeting with Amity
Time: Sep 19, 2019 08:00 PM Pacific Time (US and Canada)
Join from PC, Mac, Linux, iOS or Android: https://stanford.zoom.us/j/872623637235375142
Or iPhone one-tap (US Toll): +18333021536,,872623637# 235375142# or +16507249799,,872623637#235375142#
Or Telephone:
Dial Dial: +1 650 724 9799 (US, Canada, Caribbean Toll) or +1 833 302 1536 (US, Canada, Caribbean Toll Free)
Meeting Meeting ID: 872 623 637235 375 142
International
International numbers available: https://zoom.us/u/acbh6sIItVabCOWPniNq
Meeting Meeting ID: 872 623 637235 375 142
SIP: 235375142@zoomcrc
SIP: 872623637@zoomcrc.com
...
Minutes
Bold face refers to new items discussed at the meeting, normal face is background to assist in undersanding.
Deploying Android PingER
The Android version of the PingER MA, is described with comments at ePingER on Android Native - Amity project (this a proposal/description from Aayush Jain)
- Amity's documents describe a multipurpose, stand-alone device that can be widely distributed, something that we have brainstormed about for a long time.
- Bebo mentioned it to Topher (the Principal Investigator (PI) of the RainForest project) and he feels that the Amity PingER/Android app (when completed and vetted by Topher’s team) could easily be installed as a default service on his rainforest monitors (certainly future ones, not devices already in place). Merging the service data that he already collects with that unique to PingER has the potential to lead to some interesting results.
- Since the above Topher has been consumed with RainForest projects and will not have time to devote to PingER/Android.
- A question is whether he can provide Android's and is this needed anyway? Bebo indicated he will get hold of Androids.
The topics are: :
- How should we proceed?
- We Back in January 2019, we (Topher and the PingER team) agreed to request Amity to share the App and instructions with us; we will look at installing on a jailbroken Android phone at the San Francisco end and try it out.
- Instructions from Aayush to load the application are at ePingER Functional prototype (for Androids only)
- We also need to finish up the proxy acquisition of data from the Android MAs. A first step is outlined in https://confluence.slac.stanford.edu/display/IEPM/Proxy+support+for+PingER.
- A later step is to look at, evaluate and find resources to implement the ideas expounded in the Future Proposals section of the paper Amity is working on (is there a more recent copy)
- We Back in January 2019, we (Topher and the PingER team) agreed to request Amity to share the App and instructions with us; we will look at installing on a jailbroken Android phone at the San Francisco end and try it out.
- Amity demonstrated the Android/ePingER application via a shared screen.
- Currently they can In January they could upload the data from the App to an FTP server at Amity.
- Given concerns about security, there was a discussion about replacing with something a bit more modern such as using a centralized cloud so all can access the data. Amity are looking at Firebase:
- Firebase Application: The need for a proxy server can be completely eliminated by shifting to a cloud-based architecture for managing files. Instead of SLAC pulling in the generated txt files from MAs around the world, the MAs can themselves push these files to a centralized application server hosted on the cloud; which makes it easier for SLAC to access files as per their need. Considering the great degree of integration capabilities Google offers with its products, the flexibility arising from using the Google Cloud Platform would be pronounced. Google’s Firebase is a mobile and web application development platform that provides developers with a variety of tools and services to help develop high-quality apps, that can scale easily as per changing demands, and which delivers 99.99% uptime. The Firebase SDK allows mobile app developers to quickly add critical and reliable functionality to their applications in a short time. The recommended option here would be to leverage the Cloud Storage for Firebase that allows robust uploads and downloads onto the Google Cloud Storage buckets. Apart from the user authentication module that comes bundled with Firebase, developers can also declare file security parameters so as to allow only certain filetypes to be uploaded. Having all user uploaded files in one place will then enable SLAC to access and process files as and when needed. Server-side processing can be done on the Google Cloud Platform as well, thereby eliminating any need for SLAC to maintain its own physical infrastructure. This form of implementation can be highly beneficial in any kind of region of the world, also including remote places, like deep inside tropical rainforests, or places that have recently been hit by a natural calamity. As this implementation model is not dependent on any local measuring agent, the Android mobile apps can directly deliver data to SLAC over any form on internet connection in minimal time with high reliability.
- For At the January meeting we agreed that for the initial stages, FTP is sufficient, the Firebase model allows much greater scalability, especially as the number of Android/ePingER MAs increases.
- The Firebase archiving is now working. It supports authentication with a userid. It is accessible via the web. Once one has signed in you can use the app.
- The testing has 4 or 5 Beacons. Beacons are static.
- The MA will support mobility.
- The idea is to add the location (lat/long) of the Android MA to the data records, e.g. by putting the lat/long at the end of the current records separated by a semi-colon (';').
- There is a Google API to get the Lat/long.
- Amity plan to update the records with the lat, long data in a week. They will mail the update with the lat, long included.
- Access is by WiFi or the phone.
- The phone will cost by volume of data transferred.
- The app is a native App and available from the Android store (Play store). It has been vetted by Google.
- If the target is mobile then the MA and the PingER database will need to be extended to use the phone number of the client since it will probably not have a permanent IP address.
- There is currently no talk of adding the lat, long of the beacons
- The MA will support mobility.
- This would probably require adding GPS information (Lat/Long) to the actual measurements.
- This would be easy to implement in the MA.
- To be backward compatible we would need to make it easy to ignore this extra information.
- A concern raised was that cell mobility results in huge changes in RTTs. The current PingER analysis is based on static MAs and is typically based on looking at changes in performance based on more global and temporal influences such as social unrest, earthquakes, tsunamis, economics, education, development etc. Understanding the effects of mobility would require new analyses.
- The consensus was that, at least in the near term, the current PingER dataflow/analysis would not change. It would/could use the data from Android/ePingER static MAs.
- This implies the current PingER dataflow/analysis is extended to identify mobile (non-static) MAs, and hence ignore them or treat them differently. Les does not think this is difficult to incorporate once we decide on the format etc.
- Given concerns about security, there was a discussion about replacing with something a bit more modern such as using a centralized cloud so all can access the data. Amity are looking at Firebase:
- Actions:
- Amity will annotate the image of the Amity participants (see image above) with their names. Les sent an email reminder 1/5/2019.
- Amity will send SLAC the final copy of the paper to be presented at a Confluence conference 1/11/2019 entitled. Les sent email reminder 1/5/2019.
- It would also be helpful if Amity would set up an email list for the Amity team and share the information. A possible name might be pinger-am
- Topher will look at installing the Android App on an Android phone at the San Francisco end and try the Android/ePingER application out.
- Topher asked for which PingER web sites to look at for understanding PingER. Les later sent the following:
data aStart at the PingER home page at http://www-iepm.slac.stanford.edu/pinger/. Some important links are:
Various Reports (under the Reports tab) in particular
- How Bad is Africa's Internet
- IEEE Paper 2000
- Mapping the Digital Divide
- 2017 Annual Report
- A Giant Leap in 2016
Case studies
For users who want to look at the data see Results/data tab,.
Introductory stuff
- Under About tab
Under Maps
- Website map (mainly for developers)
- Les will work with Amity to see if they are going to add GPS information to the MA measurements and if so to understand the format and how to ignore them in the PingER dataflow/analyses.
- Les will contact Amity to see how to collect their Android/ePingER data. Les sent email 1/5/2018.
Status of Amity MA
The Amity MA is unreliable so using it for a case study does not appear fruitful. A start on such a paper has been made. Les is working with Amity to try and understand this unreliability (emails 10/6/2018, 12/6/2018, 12/23/2018), they say "There is an internet problem we are taking care of it and will get back soon".The ability to gather data from it is very intermittent, It runs fine for several days, then there is no data for many days.
For example:
...
- Some thoughts on mobility can be found at PingER and mobility.
- There was a request from Amity on how to access the SLAC data for Case Studies.
- Les sent information to Amity on this. The information is also at PingER Accessing the data.
Saqib
Saqib has submitted and had accepted a concept note “ePingER - Android based Internet Performance Monitoring Agent to measure & analyse the Digital Divide in remote areas of Pakistan.” He is now working on the next stage of the proposal. There was a comment from the evaluators:
- It would be good to prepare how to ensure that there is a beneficial effect as a result of making these measurements, and what is the practical outcome for people in the country.
Saqib will look at comparing both a native App and a perl based one. Obviously the low power requirements and possibility of using solar power and batteries is very important for remote areas. The benefits are identifying and understanding the network performance, response time, ability to make phone calls, diurnal variations, reliability, data transfer rates expected. Sharing this information and recommendations with the carriers should assist with identifying what upgrades are required and improve the ability or the local people to communicate with the outside world. Improved communications should assist in knowing when and where to market goods, share information among the community, access information from elsewhere etc.
The full and final proposal should be sent to asiaconnect-call@teincc.org by 23:59 KST (GMT +9:00) 6th November.
Jammu Kashmir
Given the unrest in Jammu Kashmir Les added a couple of targets in Jammu Kashmir. The idea is to see if the unrest is reflected in the ping performance. The hosts in JK are :
| Project Type D means it is Disabled, no longer being monitored (since it is only a few msec from SLAC and hence a proxy that is not in JK), so we only have 2 targets in JK | ||||||||||||||||||||||||||||||||||||||||||||||||
| ||||||||||||||||||||||||||||||||||||||||||||||||
The two hosts are shown in the map below:
Status of Amity MA
The problem with Amity MA reliability continues. Why is it unreliable, can anything be done?
The problem is that the utility power source goes down each night.
The MA is a desktop. It probably draws over a 100 Watts, thus a UPS for say 12 hours would be very expensive.
A possibility might be to port it to a low power device such as an Android or Blackberry Pi. They draw a few watts, so a smaller cheaper UPS might do the job. Some thoughts on this can be found at ePingER on Android phone.
Actions:
- Amity will mail the update with the lat, long included.
- Amity will annotate the photo captured of the Amity team in the meeting.
- SLAC also needs to know how to access the Firebase data.
- Once that is known and the data with lat, long included and we understand the format and how to access it, Les will look at how to incorporate in a backward-compatible fashion into the current PingER.
- Once that is known and the data with lat, long included and we understand the format and how to access it, Les will look at how to incorporate in a backward-compatible fashion into the current PingER.
- Bebo will get hold of some Android phones and once he is given instructions on accessing the native app look at experimenting.
- Amity will look to see if they know of any targets in Jammu Kashmir.
- Next meeting will be in a month.
- Amity will set up a google poll to determine the date and time
- The current time of 20:00 in the US and 08:30 at Amity is appropriate.
- Amity will set up and notify the team of the meeting by Zoom. If this is a big problem Les can set up the Zoom meeting.
- Amity will set up a google poll to determine the date and time
| Code Block |
|---|
1545191157,12/19/2018 3:45,320.544,320.908,321.415,0
1545192994,12/19/2018 4:16,0,0,0,100 ...
and ended:
1545376588,12/21/2018 7:16,0,0,0,100 1545378366,12/21/2018 7:46,319.922,320.940,321.961,0 |
Since we gather data around 1am PST or 9am GMT this means we were unable to gather data from Amity on 12/19/2018 and 12/20/2018.
- Again we went back and manually collected what we could of the measurements made at Amity MA. Looking at the table of pings from pingeramity.in to www.ncms.ae, we see missing measurements starting just after 3:25am 12/19/2018 until 14:22 12/20/2018.
The main reliability problem was that the IP address was dynamic. It has now been changed to static. There has also been a power problem so they will replace the UPS next week.
...