Thursday, June 7th 9 pm Pacific time; Friday, June 8th 2018 9:00 am Pakistan time; 12:00 noon Malaysian & Guangzhou time; and 11 am Thailand time.
New items and updates are in bold face.
See: http://pinger.unimas.my/pinger/contact.php
Need to add Umar - Johari
Wajahat Hussain (SEECS)-, Saqib (GZHU)+; Johari (UNIMAS)?; Adib (UUM)+; Fizi Jalil (MYREN); Dr. Charnsak Srisawatsakul (Ubru)+, Les+, Bebo+(in UK 8hours ahead of Pacific Summer time), Umar+
+ Confirmed attendance
- Responded but Unable to attend:
? Individual emails sent
Wajahat, Saqib, Adib, Charnsak, Umar, Bebo, Les
Administration
Looking into moving PingER to a "blockchain" database good for decentralizing distribution of data. Monitoring sites would then be able to write to a distributed ledger. This would change the architecture to a more peer to peer architecture. It helps with continuity of PingER since reduces dependence on a single site (SLAC). See Block Chain in Future PingER Projects. Bebo sent several references to Saqib who has looked at them. We could start with real-time data without including the whole archive, i.e. in parallel to the continued centrally managed archive. It would be a private Blockchain and hence not be as compute intensive as a public blockchain. Bebo's impression is that Saqib will lead in putting the ides in his paper into practice. Saqib will need some students. Saqib's boss is going to the NY City meeting.
For his IPv6 monitoring Charnsak is using ~ 100 IPv6 targets from Saqib. They are in the <HostList> of the Ubra PingER MA. It can be seen from http://www-wanmon.slac.stanford.edu/cgi-wrap/pingtable.pl?file=average_rtt&by=by-node&size=100&tick=monthly&from=TH.UBRU.CS.AC.PINGER6&to=WORLD&ex=none&only=all&ipv=all&dataset=hep&percentage=any. It is still running smoothly.
Charnsak is looking at a host in Chan Parsa province in Laos as a potential site for a PingER MA. Charnsak just got approved to make contact with the Champasak University. He expects to set up the MA in the next 4-5 months. It also depends on the partner university.
Charnsak would like to have write access to parts of the PingER Wiki site. Les investigated and it appears this can be done, even if Charsack does not have a SLAC account. Les sent Charnsak the relevant information after the previous meeting. Charnsak filled out the form and submitted. However, it went to the wrong person for approval. This has now been fixed and he should be able to move forward. Charnsak has tested and it is working.
Regarding the paper " Socioeconomic Development indices and their Reflection on Internet Performance in the ASEAN Countries ", Adib tried the Elsevier Telecommunications journal. They responded and Adib commented:
insufficient understanding of the complexities in the causality argumentation and related findings.
Adib responds: we have addressed IEEE access reviewers' comments carefully. complexities and even validation can be further enhanced, but our data is a bit limited, we have only ten values for each index. We have no data available for ALL indices in 2017 (the year this study was conducted), not even 2016. Values of all social economic indices are only available in 2015. I would suggest considering this point in our next paper, where we can update our internet performance data to 2018, and hopefully other indices data are available as well.
the relevant previous literature is not well addressed
Adib responds: Actually, we reduced LR session to minimize the paper size so it can meet the journal requirements. Adding more LR can be done, but then the size will be increased again. In fact, not many close and recent papers were published in this particular area. so it is not necessary to add more LR, especially we are not comparing our final model with them.
As a minor but still disturbing deficiency, references to the cited papers are incomplete.
Adib agrees and it is already amended
Adib will submit the paper to World Development: https://www.journals.elsevier.com/world-development
We were unable to gather data from pinger.uum.edu.my. It appears to have been fixed.
NUST: (Updated 5/3/2018))
The web page http://maggie.seecs.nust.edu.pk was not working, email sent to Wajahat 6/3/2018, it has been re-discovered and is working
No intern has joined Wajahat's Lab. So there is not much progress.
Wajahat proposes to get a list of the new Universities in Pakistan and contact them encouraging them to participate in PingER and set up MA. They have made a list of new university sites, communications networks, Labs in different regions of Pakistan (especially the remote regions) and will make contact.
Wajahat is hoping to get few students to work on Master thesis related to pinger data. If possible kindly share ideas related to data analytics which he can share with students
Wajahat also asked:
Saqib submitted the Camera ready paper on “A Blockchain-based Decentralized Data Storage and Access Framework for PingER” and it has been accepted in Trustcom2018.
Two previously accepted papers are now online on the following links.
IPv6 node in Beijing is up and running for the last 7 weeks but not available from outside China. It does not appear possible to make the host accessible outside China. We are looking at alternates (e.g. using the anonymous FTP server at SLAC as a proxy) that may also be relevant for PingER on Android gathering of data.
Saqib sent an email to the team:
In early years of PingER, the framework was designed to check the latency and other Internet performance metrics between CERN and SLAC to facilitate the data transfer between the two sites.
I am thinking, is there any possibility to use PingER to monitor the health of the Bitcoin blockchain network? Since the latency is critical in Bitcoin blockchain network as all the incentives depend on the propagation of transactions and mined blocks. Thus, I am only interested in measuring the latency to check its effect on the propagation of the transactions and mined block among different mining pools. I think if we can do such thing on a historical basis as PingER already does for the Internet, it will increase the worth of the framework and its usability.
Maybe a few test experiments can guide us to a good research paper. I am not sure about the feasibility idea, therefore, need your kind feedback
Umar responded:
Umar looking at extending the comparison IPv6 vs IPv4 ping RTTs and TCP vs ICMP/ping RTTs.
Y. Zhang, N. Duffield, V. Paxson, S. Shenker, On the Constancy of Internet Path Properties, in ACM SIGCOMM Internet Measurement Workshop 2001
A. Acharya, J. Saltz, A Study of Internet Round-Trip Delay, Univ of Maryland, Tech Rep. CS-TR-3736
M. Allman, V. Paxson, On Estimating End-to-End Network Path Properties, in ACM SIGCOMM 1999
V. Paxson, G. Almes, J. Mathis, Framework for IP Performance Metrics
IPv6 results gathered using ping-vs-tcp.pl script. About 56 nodes with IPv6 addresses, 14 of which responded with Npings
PingER IPV6 support
Meeting with Amazon Web Services
Host | State | last seen | Status |
---|---|---|---|
pinger-ncp.ncp.edu.pk | pinger-ncp.ncp.edu.pk down | Nov 29, 2017 | |
121.56.146.180 (pinger.kohat.edu.pk) | Down | Nov 22nd, 2017 | |
cae.seecs.edu.pk | Down | Feb 27, 2018 | |
pinger.isra.edu.pk | Down | March 6, 2018 | |
pinger2.if.ufrj.br | Flaky, down since 4/28/2018, and out of disk space, pinger.xml truncated, emaile, sent email 4/28/2017, they are looking at it | April 4/28/2018 | Fixed 5/26/2018. |
pingeramity.in | It is partially working. Now working on missing beacons.txt file and missing data (i.e data disappears a few hours after it is measured and saved at MA). Also it is unable to access http://www-iepm.slac.stanford.edu/pinger/beacons.txt | April 27, 2018 | |
pinger.uum.edu.my | Unable to gather data since 5/26/2018. Unable to ping. Sent email to Adib 6/1/2018. Appears to be working again 6/6/2018 | May 26/2018. | |
netmon.physics.carleton.ca | Unable to gather data, host is pingable, emailed contact 6/4/2018. Fixed 6/6/2018. | June 1st, 2018 | |
pinger.unimas.my | Unable to gather since Feb 26, 2018, Disabled in April 2018. Working again 6/6/2018, re-enabled | Feb 26, 2018 | Working again. |
Next meeting: Thursday, July 5th 9pm Pacific time; Friday, July 6th, 2018 9:00am Pakistan time; 12:00noon Malaysian & Guangzhou time; and 11am Thailand time.
There is an upcoming grant call for projects between Pakistan and the US. Topics may be focused on cybersecurity, health, and education. It has not been announced yet. Wajahat will get the details and share them with the team as soon as they are available. It is interesting since getting a US partner appears to be a roadblock for many potential Pakistani responders. However, the topics may not be very related to PingER. NUST is looking at applying to set up a cyber lab. Getting the funding will be in competition with other Pakistani Universities. For cyber the main things we could think of from PingER were: quantifying what fraction of hosts block pings, punching holes in firewalls to allow pings, how to misuse ping (e.g. ping-of-death, or using anomalous ping packets to deduce the OS etc. flood pings for DOS), the host can respond to ping but applications do not work. Fear of misuse of pings can result in the system administrator, network administrator or cybersecurity blocking pings. A possibility might be a study of what fraction of say working www/dns etc. apps (i.e. checking if a host responds to the relevant port) do not respond to pings. This could be by application, by country or by region etc. Also how to protect a remote pinger traceroute or server from being used in DOS attacks. As of 3/27/2018 there is no call so far. There was one last year, so Wajahat is expecting one. Emailed Wajahar 6/3/2018 asking for update. He responded "There is no call yet. There was one last year. May be change in the US-Pak policy. Just a guess."
Adib, Bebo, Les met with Southampton Web observatory person. There seemed to be enthusiasm. Adib was going to send some materials to Southampton. The person at Southampton gave us some links. Adib is in the early stages of exploring what web observatory data to link with such as business context indicators, social media and government sites. There was no update 3/29/2018, or 5/3/2018.
Saqib submitted a project in CERNET to monitor the performance of IPv6 network using PingERv6. He received the news that the project is accepted with 100K RMB. Now he has 2 accepted projects regarding PingER and total amount he has is near about 40K USD. Further, in his lab, three U1 servers have already arrived through another grant for research purpose. We can also use them for our PingER project.
Therefore, the CERNET has given Saqib a IPv6 based CentOS 6.8 machine in cloud. Now he is trying to deploy the PingER server on the machine. Let's see how it will work on IPv6 based network. This is a 2-year project.
Saqib has made contact with John Pickard author of "Quality of IPv6 Enablement of Universities: An International Study" who has provided a list of about 125 Universities in about 60 countries hosting IPv6 sites. However many are proxies. Les has suggested using perfSONAR (there are about 1000 and they all have lat longs in the perfSONAR database. Saqib is gathering the list, then we will see how many have IPv6 addresses.
The paper title: " Missing Values Imputation in PingER Internet End-to-end Performance Measurements using k-nearest neighbors (k NN)" was not accepted in IMC 2017. He is updating the paper according to the reviewer’s comments. Hopefully, Saqib will submit it at some other venue. Not yet decided on the submission venue. Need some suggestions. Updated but not decided where to submit. Update 12/4/2017?
Currently, no data is available on PingER on Android due to unavailability of the live IP address. No update 4/19/2017, 7/6/2017. Email sent to Sara Masood. No update 9/24/2017. Any update 10/24/2017. No progress 1/18/2018.
PingER has valuable historical data for the last 20 years.Many analysis and case studies have been carried using this data. A lot of information is available on the website. Saqib's idea is to publish the brief summary all these analysis through a survey paper covering the history and utilization of PingER data starting from 1998 to 2017. Saqib started on it, Les is providing assistance. Need your feedback on the idea of Measuring the Digital Development of the Countries using PingER data. Is there something you want me to review some, e.g. some draft document on Measuring the Digital Development of the Countries using PingER data , or are you asking if it is a good idea to review and create such a document. If the latter I think this is a fascinating subject. Part of the challenge is the chicken and egg problems: i.e. is it network performance influencing advancement of the country, or is it the reverse that advanced countries can afford good networks. My belief is it goes both ways. Also one needs to extend the analysis beyond just Africa else it’s kind of a repeat of Pinging Africa , R. Les Cottrell, IEEE Spectrum February 2013. Also see A Simple Tool for Measuring Digital Development , by R. Les Cottrell, IEEE Spectrum February 2013. This is derived from SLAC-PUB-15333.
"BIND: An Indexing Strategy for Big Data Processing" that uses PingER data. Submitted and accepted by the 2017 IEEE Region 10 Conference (TENCON) that takes place in November. In Penang Malaysia
The paper title: "Detecting Anomalies from End-to-end Internet Performance Measurements (PingER) using Cluster Based Local Outlier Factor" is submitted in ISPA 2017 (http://trust.gzhu.edu.cn/conference/ISPA2017/). It has been accepted as of 9/17/2017.
The thesis of Aqsa Hameed title “Applying Data Mining and Visualization Techniques on Pinger Data” is published in ODBMs.org and is accessible through http://www.odbms.org/2017/07/applying-data-mining-and-visualization-techniques-on-pinger-data/
Preparing a paper on the impact of the cyclone Verdha that hit the Indian coast along with many countries like Thailand, Sri Lanka, Malaysia, Maldives on December 6th. They use K-Means clustering (see https://en.wikipedia.org/wiki/K-means_clustering) to identify anomalies in packets received (inverse of loss) and maximum RTT. Note that for December 7th the reduction in packets received.
From: Aayush Jain <aayush.2896@gmail.com>
Sent: 24 March 2017 12:31
To: A. Sai Sabitha; harysinha@gmail.com
Subject: PingER Android Team
Progress Made So Far
So far Shivnarayan Rajappa and Rohan Sampson’s team have succeeded in making a bare-bones Android Application that can ping beacons, parse data, and generate a text file in a format specified by SLAC ready for uploading. The proposed model involved the application pulling the beacon list from SLAC’s servers for pinging. However, the present application has a small percentage of the beacon list hard-coded into the application. As of now, the link between the application and proxy server has not been established.
Future Plans
The new team members are:
1. Rohit Raj
2. Shagun Seth
3. Savy Gupta
4. Aayush Jain
5. Tanuj Saraf
Owing to the advancements in Android technologies during the time of development of the project, our team believes that we can create a more capable and robust application for this project. This involves rewriting the entire application from scratch.
We also propose to create a proxy server that can act as an intermediary between the Android application and SLAC’s servers. The proxy server would thus allow handling multiple hosts for greater data collection.
Approach
Our team plans to start off by completing the work on the Android app within 20 days. We will recreate the entire app, with an improved workflow for greater stability. The app will parse the beacon list from SLAC’s servers and save as an XML on the device. The data generated after every ping will be appended to a file after cleaning it up with RegEx matches. We first plan to test the app with only a few members of the ping list (which will later be expanded to auto-update in its entirety).
Once we accomplish our work with the app, we will move on to the task of establishing a proxy server. Our entire team will focus on the components of networking, host management, host authentication, file synchronization, and security.
By the end of the project, the server will be able to handle multiple hosts which would all forward it data, and it would in turn reorganise it again for SLAC’s servers to pull.
The students successfully presented the paper on the PingER implementation on Android.at the confluence 2017 conference.. The paper is submitted to IEEE section.
Tropical cyclone Vardah hit Chennai in India on the Dec 13th. It impacted the Internet, in particular one of Airtel's undersea cables. Les sent email to A. Sai Sabitha to see if PingER from Amity could see any effect.
During the next 6 months their research will study the impact Vardha cyclone that hit the Indian coast(South India/Chennai) and a few other neighboring countries in December 2016 as seen bu PingER.
The idea is to study and analyze the PingER data during the corresponding time frame and deduce significant trends and patterns from the data using
1. Clustering techniques
2. Time series
3. Correlation and Regression concepts
They have fixed the final sequence number change by using regex, and pushed these changes to github repository.
They have installed apache tomcat in the server and plan to use a java file on the server which would connect to the phones that send the request. This java file will then take the input stream received from the phone and write the output stream to a file that would be stored on the server. We are facing some problems regarding a blocked port that is not allowing the phone to connect to the server we are currently working on resolving the issue.
SLAC can then regularly pull these files which would be stored based on the month they are received.
The Android students have started writing a paper on " implementation of pinger on android " .
Next steps:
Extend the target list by getting the Beacon list from SLAC. It is at http://www-iepm.slac.stanford.edu/pinger/pinger.xml on a regular basis and updating the <BeaconList> section at their site. This was part of pinger2.pl.
To a large extent it depends on how we plan to use this.
A next step is to get the data from the phone MA to the archive at SLAC. The current method ping_data.pl requires a public IP address for the phone which may not exist if its is mobile. Getting the MA to put the data to the archive may raise some security issue for the archiver.
Need your feedback on the idea of Measuring the Digital Development of the Countries using PingER data
Two days ago we started being unable to gather data from pinger.fsktm.um.edu.my (103.18.2.152). When one tries ping it fails,
ping pinger.fsktm.um.edu.my
ping: unknown host pinger.fsktm.um.edu.my
Exit 2
However pinging the IP address works:
117cottrell@rhel6-64i:~$ping 103.18.2.152 from http://202.28.194.4/toolkit/gui/reverse_traceroute.cgi?target=pinger.fsktm.um.edu.my&function=traceroute
PING 103.18.2.152 (103.18.2.152) 56(84) bytes of data.
64 bytes from 103.18.2.152: icmp_seq=1 ttl=48 time=265 ms
64 bytes from 103.18.2.152: icmp_seq=2 ttl=48 time=266 ms
64 bytes from 103.18.2.152: icmp_seq=3 ttl=48 time=265 ms
64 bytes from 103.18.2.152: icmp_seq=4 ttl=48 time=265 ms
^C
--- 103.18.2.152 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3277ms
I thought it might be our DNS resolution, however I also cannot see it from Thailand, i.e. from
It gives
Can't find IPv4 address for host name pinger.fsktm.um.edu.my. Probably an unknown host.
I get the same result from a host in Pakistan http://comsatsswl.seecs.edu.pk:8080/cgi-bin/traceroute.pl?target=pinger.fsktm.um.edu.my&function=traceroute