- The platform requires the installation of client software, which is similar to Skype. As with other conference call software, it allows us to share video and screencasts. (Umar promises that Zoom doesn't pay him to market their software
)
This meeting: Monday, November 19th 8 pm Pacific time (Nb now on winter time); Tuesday, November 20th, 2018 9:00 am Pakistan time; 9:30 am India time; 12:00 noon Malaysian & Guangzhou time; and 11 am Thailand time; 6:00am Jordan time (I think)
New items and updates are in boldface.
See: http://pinger.unimas.my/pinger/contact.php
Wajahat Hussain-, Taha (SEECS), Saqib+ (GZHU); Johari+ (UNIMAS); Adib (UUM); Dr. Charnsak Srisawatsakul (Ubru), Les+, Bebo+, Umar+
Saqib (but we were unable to hear him), Dr Shadi Aljawarneh, Umar, Les, Bebo
Administration
Umar sent Shadi two URLs on how to download and install the MA software and possible interests for students.
Charnsak is looking at a host in Champasak University, Chan Parsa province in Laos as a potential site for a PingER MA. Charnsak just got approved to make contact with the Champasak University. He expects to set up the MA in the next 4-5 months (say towards end 2018). It also depends on the partner university, and there may be a lot of paperwork.
Need to add Umar Kalim to http://pinger.unimas.my/pinger/contact.php. From the 7/5/2018 meeting: Johari can't ssh into the server so he will go to it on Monday. He will also upload the new UNIMAS PingER website next week.
Sent reminder emails 8/6/2018. 9/3/2018.
Les has sent Adib updates to Figs 3, 4, 5 to extend out to 2018. This is for the paper Socio-economic Development Indices and Their Reflection on Internet Performance in ASEAN Countries
Adib will submit the paper to World Development: https://www.journals.elsevier.com/world-development
NUST: (No update 11/19/2018)
Wajahat proposes to get a list of the new Universities in Pakistan and contact them encouraging them to participate in PingER and set up MA. They have made a list of new university sites, communications networks, Labs in different regions of Pakistan (especially the remote regions) and will make contact.
comsatsswl.seecs.edu, maggie1.seecs.edu.pk, maggie2.seecs.edu.pk, monitor.seecs.edu.pk,
pinger.isra.edu.pk (no data since 5th Sept 2018, so it is a candidate to Disable),
pinger.nwfpuet.edu.pk, pinger.vu.edu.pk
Saqib's future at GZHU will be much clearer after November, current contract expires February 2019.
We were unable to hear Saqib.
After the meeting, Umar and Saqib experimented with using Zoom instead of Skype. Saqib commented that the call quality was significantly better than that of Skype. Umar could hear him clearly. Therefore, Umar suggests that we should schedule a Zoom call for our next meeting. He will make the arrangements of scheduling and hosting the call.
)
PingER at SLAC (Updated 11/19/2018)
PingER IPV6 support
| Host | State | last seen | Status |
|---|---|---|---|
| pinger.gzhu.edu.cn | Is not working | 10/20/2018 | |
| sitka.triumf.ca | Host responds but no data gathered. Email sent to sitka contact, machine is old and suffered a power outage, flaky after recovery, ran pinger2.pl from command line. May be a problem with /tmp/ directory. Look at replacing with a VM. Machine rebooted and all is OK | 10/27/2018 | Fixed as of Oct 30th, 2018. |
| pinger.unesp.br | MA is working but it can only ping itself, sent email 10/30/2018, last good data was 10/25/2018. The MA can only ping itself. Emailed 10/30/2018. | 10/25/2018 | Fixed as of GMT: Tuesday, October 30, 2018 5:40:15 AM |
| pingersonar-um.myren.net.my | No response | 6/26/2018 | |
| pinger.isra.edu.pk | Down, it came up Sep 2-4, 2018. It is pinging, however, all the targets are not responding after 31 tries, email sent to Wajahat 9/18/2018. As of 10/14/2018 it is not pingable. | March 6, 2018 | |
| pingeramity.in | It has been working since 28th July. It is unclear how stable it is. It is down again 9/5/2018. Worked again from 9/24/2018 to 9/28/2018. Emailed Amity requesting clarification 10/3/2018. They are looking at it. | April 27, 2018 |
Context:
Questions:
Next meeting: There is a Doodle poll sent 12/13/2018. December week of 10-13: 10 pm Pacific time (Nb now on winter time); a day later 11:00 am Pakistan time; 11:30 am India time; 2:00 pm Malaysian & Guangzhou time; 1:00pm Thailand time; 8:00am Jordan time
Blockchain
IPv6 measurements
We need to think about how to handle these hosts with both IPv4 and IPv6 addresses. We could do nothing, the simplest solution. Or we could move towards moving to the future and simply change the IPv4 addresses to IPv6 addresses. In a few cases, we may be able to find two hosts at a site or two names for the same host (i.e. an A and a CNAME record), using an IPv4 address on one and an IPv6 on the other. This is the next simplest solution. Doing this loses having a historical record of a target with both an IPv4 and IPv6 address. Umar and Les have some scripts that enable comparisons to be made between access to a host via its IPV4 and its IPv6 address without using the PingER data. An alternative would be to also have a pseudo name for such ambiguous hosts, another would be to modify the database schema. Both of the latter two would require changes to the code in several places. Another alternative might be to create a second record using either the IPv4 or IPv6 address as the name. This requires no changes to the code but is pretty unpleasant. If one could find a CNAME as well as the regular name (A reg) then one could use the CNAME for one of the addresses and the regular name for the other address.
The Android version of the PingER MA,is described with comments at ePingER on Android Native - Amity project (this a proposal/description from Aayush Jain)
It describes a multipurpose, stand-alone device that can be widely distributed, something that we have brainstormed about for a long time.
We agreed to request Amity to share the App and instructions with us; we will look at installing on a jailbroken Android phone at the San Francisco end and try it out.
They are working on a paper. They sent a 1st draft they have sent to Bebo and Les that we responded to 10/6/2018, a 2nd draft was responded to by Bebo and Les 10/13/2018. It has been submitted to EasyChair.
They will be attaching the android app for review after testing it again.
Topher feels that the Android app (when completed and vetted by his team) could easily be installed as a default service on his rainforest monitors (certainly future ones, not devices already in place). Merging the service data that he already collects with that unique to PingER has the potential to lead to some interesting results.
Also August 16th and again September 3rd, then October 6th and October 13th proposed a meeting between: Bebo, Umar and Lesand the Amity folks.
Y. Zhang, N. Duffield, V. Paxson, S. Shenker, On the Constancy of Internet Path Properties, in ACM SIGCOMM Internet Measurement Workshop 2001
A. Acharya, J. Saltz, A Study of Internet Round-Trip Delay, Univ of Maryland, Tech Rep. CS-TR-3736
M. Allman, V. Paxson, On Estimating End-to-End Network Path Properties, in ACM SIGCOMM 1999
V. Paxson, G. Almes, J. Mathis, Framework for IP Performance Metrics
IPv6 results gathered using ping-vs-tcp.pl script. About 56 nodes with IPv6 addresses, 14 of which responded with Npings
Saqib sent an email to the team:
In early years of PingER, the framework was designed to check the latency and other Internet performance metrics between CERN and SLAC to facilitate the data transfer between the two sites.
"I am thinking, is there any possibility to use PingER to monitor the health of the Bitcoin blockchain network? Since the latency is critical in Bitcoin blockchain network as all the incentives depend on the propagation of transactions and mined blocks. Thus, I am only interested in measuring the latency to check its effect on the propagation of the transactions and mined block among different mining pools. I think if we can do such thing on a historical basis as PingER already does for the Internet, it will increase the worth of the framework and its usability.
Maybe a few test experiments can guide us to a good research paper. I am not sure about the feasibility idea, therefore, need your kind feedback"
Umar responded:
More and more I am convinced that we could identify an interesting and relevant project taking advantage of our PingER experience and involving Blockchain technology.I also think that it may be possible for us to get funding/grant money for such a project which might clearly draw the attention of Blockchain entrepreneurs, conferences, and publications. It might also allow us to attract the collaboration of other institutions and/or universities that might otherwise have not been interested in the basic PingER goals and technology.
Bebo's impression is that Saqib will lead in putting the ideas in his paper into practice. Saqib will need some students. Saqib's boss is going to the NY City meeting.
Saqib submitted the Camera ready paper on “A Blockchain-based Decentralized Data Storage and Access Framework for PingER” and it has been accepted in Trustcom2018.
Two previously accepted papers are now online on the following links.
There is an upcoming grant call for projects between Pakistan and the US. Topics may be focused on cybersecurity, health, and education. It has not been announced yet. Wajahat will get the details and share them with the team as soon as they are available. It is interesting since getting a US partner appears to be a roadblock for many potential Pakistani responders. However, the topics may not be very related to PingER. NUST is looking at applying to set up a cyber lab. Getting the funding will be in competition with other Pakistani Universities. For cyber the main things we could think of from PingER were: quantifying what fraction of hosts block pings, punching holes in firewalls to allow pings, how to misuse ping (e.g. ping-of-death, or using anomalous ping packets to deduce the OS etc. flood pings for DOS), the host can respond to ping but applications do not work. Fear of misuse of pings can result in the system administrator, network administrator or cybersecurity blocking pings. A possibility might be a study of what fraction of say working www/dns etc. apps (i.e. checking if a host responds to the relevant port) do not respond to pings. This could be by application, by country or by region etc. Also how to protect a remote pinger traceroute or server from being used in DOS attacks. As of 3/27/2018 there is no call so far. There was one last year, so Wajahat is expecting one. Emailed Wajahar 6/3/2018 asking for update. He responded "There is no call yet. There was one last year. May be change in the US-Pak policy. Just a guess."
Adib, Bebo, Les met with Southampton Web observatory person. There seemed to be enthusiasm. Adib was going to send some materials to Southampton. The person at Southampton gave us some links. Adib is in the early stages of exploring what web observatory data to link with such as business context indicators, social media and government sites. There was no update 3/29/2018, or 5/3/2018.
Saqib submitted a project in CERNET to monitor the performance of IPv6 network using PingERv6. He received the news that the project is accepted with 100K RMB. Now he has 2 accepted projects regarding PingER and total amount he has is near about 40K USD. Further, in his lab, three U1 servers have already arrived through another grant for research purpose. We can also use them for our PingER project.
Therefore, the CERNET has given Saqib a IPv6 based CentOS 6.8 machine in cloud. Now he is trying to deploy the PingER server on the machine. Let's see how it will work on IPv6 based network. This is a 2-year project.
Saqib has made contact with John Pickard author of "Quality of IPv6 Enablement of Universities: An International Study" who has provided a list of about 125 Universities in about 60 countries hosting IPv6 sites. However many are proxies. Les has suggested using perfSONAR (there are about 1000 and they all have lat longs in the perfSONAR database. Saqib is gathering the list, then we will see how many have IPv6 addresses.
The paper title: " Missing Values Imputation in PingER Internet End-to-end Performance Measurements using k-nearest neighbors (k NN)" was not accepted in IMC 2017. He is updating the paper according to the reviewer’s comments. Hopefully, Saqib will submit it at some other venue. Not yet decided on the submission venue. Need some suggestions. Updated but not decided where to submit. Update 12/4/2017?
Currently, no data is available on PingER on Android due to unavailability of the live IP address. No update 4/19/2017, 7/6/2017. Email sent to Sara Masood. No update 9/24/2017. Any update 10/24/2017. No progress 1/18/2018.
PingER has valuable historical data for the last 20 years.Many analysis and case studies have been carried using this data. A lot of information is available on the website. Saqib's idea is to publish the brief summary all these analysis through a survey paper covering the history and utilization of PingER data starting from 1998 to 2017. Saqib started on it, Les is providing assistance. Need your feedback on the idea of Measuring the Digital Development of the Countries using PingER data. Is there something you want me to review some, e.g. some draft document on Measuring the Digital Development of the Countries using PingER data , or are you asking if it is a good idea to review and create such a document. If the latter I think this is a fascinating subject. Part of the challenge is the chicken and egg problems: i.e. is it network performance influencing advancement of the country, or is it the reverse that advanced countries can afford good networks. My belief is it goes both ways. Also one needs to extend the analysis beyond just Africa else it’s kind of a repeat of Pinging Africa , R. Les Cottrell, IEEE Spectrum February 2013. Also see A Simple Tool for Measuring Digital Development , by R. Les Cottrell, IEEE Spectrum February 2013. This is derived from SLAC-PUB-15333.
"BIND: An Indexing Strategy for Big Data Processing" that uses PingER data. Submitted and accepted by the 2017 IEEE Region 10 Conference (TENCON) that takes place in November. In Penang Malaysia
The paper title: "Detecting Anomalies from End-to-end Internet Performance Measurements (PingER) using Cluster Based Local Outlier Factor" is submitted in ISPA 2017 (http://trust.gzhu.edu.cn/conference/ISPA2017/). It has been accepted as of 9/17/2017.
The thesis of Aqsa Hameed title “Applying Data Mining and Visualization Techniques on Pinger Data” is published in ODBMs.org and is accessible through http://www.odbms.org/2017/07/applying-data-mining-and-visualization-techniques-on-pinger-data/
Preparing a paper on the impact of the cyclone Verdha that hit the Indian coast along with many countries like Thailand, Sri Lanka, Malaysia, Maldives on December 6th. They use K-Means clustering (see https://en.wikipedia.org/wiki/K-means_clustering) to identify anomalies in packets received (inverse of loss) and maximum RTT. Note that for December 7th the reduction in packets received.
From: Aayush Jain <aayush.2896@gmail.com>
Sent: 24 March 2017 12:31
To: A. Sai Sabitha; harysinha@gmail.com
Subject: PingER Android Team
Progress Made So Far
So far Shivnarayan Rajappa and Rohan Sampson’s team have succeeded in making a bare-bones Android Application that can ping beacons, parse data, and generate a text file in a format specified by SLAC ready for uploading. The proposed model involved the application pulling the beacon list from SLAC’s servers for pinging. However, the present application has a small percentage of the beacon list hard-coded into the application. As of now, the link between the application and proxy server has not been established.
Future Plans
The new team members are:
1. Rohit Raj
2. Shagun Seth
3. Savy Gupta
4. Aayush Jain
5. Tanuj Saraf
Owing to the advancements in Android technologies during the time of development of the project, our team believes that we can create a more capable and robust application for this project. This involves rewriting the entire application from scratch.
We also propose to create a proxy server that can act as an intermediary between the Android application and SLAC’s servers. The proxy server would thus allow handling multiple hosts for greater data collection.
Approach
Our team plans to start off by completing the work on the Android app within 20 days. We will recreate the entire app, with an improved workflow for greater stability. The app will parse the beacon list from SLAC’s servers and save as an XML on the device. The data generated after every ping will be appended to a file after cleaning it up with RegEx matches. We first plan to test the app with only a few members of the ping list (which will later be expanded to auto-update in its entirety).
Once we accomplish our work with the app, we will move on to the task of establishing a proxy server. Our entire team will focus on the components of networking, host management, host authentication, file synchronization, and security.
By the end of the project, the server will be able to handle multiple hosts which would all forward it data, and it would in turn reorganise it again for SLAC’s servers to pull.
The students successfully presented the paper on the PingER implementation on Android.at the confluence 2017 conference.. The paper is submitted to IEEE section.
Tropical cyclone Vardah hit Chennai in India on the Dec 13th. It impacted the Internet, in particular one of Airtel's undersea cables. Les sent email to A. Sai Sabitha to see if PingER from Amity could see any effect.
During the next 6 months their research will study the impact Vardha cyclone that hit the Indian coast(South India/Chennai) and a few other neighboring countries in December 2016 as seen bu PingER.
The idea is to study and analyze the PingER data during the corresponding time frame and deduce significant trends and patterns from the data using
1. Clustering techniques
2. Time series
3. Correlation and Regression concepts
They have fixed the final sequence number change by using regex, and pushed these changes to github repository.
They have installed apache tomcat in the server and plan to use a java file on the server which would connect to the phones that send the request. This java file will then take the input stream received from the phone and write the output stream to a file that would be stored on the server. We are facing some problems regarding a blocked port that is not allowing the phone to connect to the server we are currently working on resolving the issue.
SLAC can then regularly pull these files which would be stored based on the month they are received.
The Android students have started writing a paper on " implementation of pinger on android " .
Next steps:
Extend the target list by getting the Beacon list from SLAC. It is at http://www-iepm.slac.stanford.edu/pinger/pinger.xml on a regular basis and updating the <BeaconList> section at their site. This was part of pinger2.pl.
To a large extent it depends on how we plan to use this.
A next step is to get the data from the phone MA to the archive at SLAC. The current method ping_data.pl requires a public IP address for the phone which may not exist if its is mobile. Getting the MA to put the data to the archive may raise some security issue for the archiver.
Need your feedback on the idea of Measuring the Digital Development of the Countries using PingER data
Two days ago we started being unable to gather data from pinger.fsktm.um.edu.my (103.18.2.152). When one tries ping it fails, ping pinger.fsktm.um.edu.myping: unknown host pinger.fsktm.um.edu.myExit 2However pinging the IP address works:117cottrell@rhel6-64i:~$ping 103.18.2.152 from http://202.28.194.4/toolkit/gui/reverse_traceroute.cgi?target=pinger.fsktm.um.edu.my&function=traceroutePING 103.18.2.152 (103.18.2.152) 56(84) bytes of data.64 bytes from 103.18.2.152: icmp_seq=1 ttl=48 time=265 ms64 bytes from 103.18.2.152: icmp_seq=2 ttl=48 time=266 ms64 bytes from 103.18.2.152: icmp_seq=3 ttl=48 time=265 ms64 bytes from 103.18.2.152: icmp_seq=4 ttl=48 time=265 ms^C--- 103.18.2.152 ping statistics ---4 packets transmitted, 4 received, 0% packet loss, time 3277ms I thought it might be our DNS resolution, however I also cannot see it from Thailand, i.e. from
It gives
Can't find IPv4 address for host name pinger.fsktm.um.edu.my. Probably an unknown host.
I get the same result from a host in Pakistan http://comsatsswl.seecs.edu.pk:8080/cgi-bin/traceroute.pl?target=pinger.fsktm.um.edu.my&function=traceroute