Thursday October 26th 9pm Pacific time, Friday October 27th, 2016 9:00am Pakistan time, Friday October 27th 2017 12:00noon Malaysian and Guangzhou time.
New items and updates are in bold face.
See: http://pinger.unimas.my/pinger/contact.php
Wajahat Hussain+, ajmal.farooq@seecs.edu.pk, Muhammad Anas Abrar (SEECS); Saqib+, Aqsa (UAF); Johari?, Adnan Khan (UNIMAS); Badrul, Ridzuan, Ibrahim (UM); (UTM); Adib?, Fatima (UUM); Fizi Jalil (MYREN); Les+, Bebo+
+ Confirmed attendance
- Responded but Unable to attend:
? Individual emails sent
Saqib, Wajahat, Adib, Bebo, Les
Administration
Looking into moving PingER to a "block chain" data base good for distribution of data. Hot topic. Monitoring sites would then be able to write to a distributed ledger. This would change architecture to a more peer to peer architecture. Help with continuity of PingER. Might be compelling. Saqib is interested.
The paper on Internet Performance and its Reflection on Socioeconomic Development in the ASEAN Countries to IEEE Access.
IP = 2724.288 + 0.008GDP + 275.904 IDI -3238.298 HDI – 220.264 – 8.084HPI
And they test its predictive behavior using various statistical tests. It looks very good.
pinger.uum.edu.my is not responding, Adib contacted UUM IT staff, they promised to fix this issue by Tuesday. Adib will do his best to fix it this week,
Adib has sent PingER MA invitation to Nigerian person who used to be a student at UUM. He is awaiting a response.
Adib, Bebo, Les met with Southampton Web observatory person. There seemed to be enthusiasm. Adib was going to send some maerials to Southampton. The person at Southampton gave us some links.
NUST: Updates August 2017?
One student is working on: Data Mining doing research on PingER data. He will be using Hadoop on a supercomputer at NUST.
The student is getting to know Hadoop. Once the student is comfortable with the tools we can start applying the tools to pinger data. he is working on Hadoop. Student is working on exams at the monent, Wajahat will conttact student
424cottrell@pinger:~$wget 'http://pinger.vu.edu.pk/cgi-bin/ping_data.pl?'
--2017-10-26 20:01:14-- http://pinger.vu.edu.pk/cgi-bin/ping_data.pl?
Resolving pinger.vu.edu.pk... 111.68.103.43
Connecting to pinger.vu.edu.pk|111.68.103.43|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 0 [text/html]
Saving to: "ping_data.pl"
[ <=> ] 0 --.-K/s in 0s
2017-10-26 20:01:15 (0.00 B/s) - "ping_data.pl" saved [0/0]
425cottrell@pinger:~$telnet pinger.vu.edu.pk 80
Trying 111.68.103.43...
Connected to pinger.vu.edu.pk.
Escape character is '^]'.
^]
telnet> quit
Connection closed.
They are focusing on the hosts they can ping. In many cases the researcher has left and others do not see the purpose. Wajahat will keep trying pointing out the value for international research and papers.
The paper title: " Internet Performance Analysis of South Asian Countries using End-to-end Internet Performance Measurements" is submitted in IEEE IUCC 2017 (http://trust.gzhu.edu.cn/conference/IUCC2017/). Camera ready paper is submitted.
The paper title: " Detecting Anomalies from End-to-End Internet Performance Measurements (PingER) using Cluster Based Local Outlier Factor" is submitted in IEEE ISPA 2017 (http://trust.gzhu.edu.cn/conference/ISPA2017/). Camera ready paper is submitted.
The paper title: " Missing Values Imputation in PingER Internet End-to-end Performance Measurements using k-nearest neighbors (k NN)" was not accepted in IMC 2017. He is updating the paper according to the reviewer’s comments. Hopefully, Saqib will submit it at some other venue. Not yet decided about the submission venue. Need some suggestions. Updared but not decided where to submit
PingER MA at GZHU is now accessible publicly and is successfully gathering data and incorporated in PingER. It would make a good beacon, however it was not pingable 7/16/2017, sent email. Saqib is trying to convince the IT department to open the ping. No success yet. Hopefully in next 2 weeks. (9/26/2016). Still not pingable 10/24/2017. No success. However, requested to move PingER to dual stack subnet to ping Ipv6 address space. In coming week, then will see what we can ping.
We now have about 230+ educational sites in China that respond to pings, about 200 have been entered into the PingER Meta database. About 10% are not responding. Saqib has entered about 48 municipalty sites, so all sites are updated. End of this month GZHU security level should be reduced and blocked internet access goes away (since 27th August)
PingER Project of 150K RMB is accepted by Guangzhou Postdoctoral Special Funds of China. Hopefully, in the coming month, I will get two 1U servers to install data repository in GU, hopefully next month
CERNET backbone is currently at 10Gbps will upgrade to 100Gbps in a couple of years. There are about 20 core nodes that run IPv6 and they will be extending to 50 core nodes.Saqib submitted a proposal to extend PingER to support IPv6 monitoring of the CERNET backbone (deadline was 30 Sept 2017).
PingER has valuable historical data for the last 20 years.Many analysis and case studies have been carried using this data. A lot of information is available on the website. Saqib's idea is to publish the brief summary all these analysis through a survey paper covering the history and utilization of PingER data starting from 1998 to 2017. Saqib started on it, Les is providing assistance. Need your feedback on the idea of Measuring the Digital Development of the Countries using PingER data. Is there something you want me to review some, e.g. some draft document on Measuring the Digital Development of the Countries using PingER data , or are you asking if it is a good idea to review and create such a document. If the latter I think this is a fascinating subject. Part of the challenge is the chicken and egg problems: i.e. is it network performance influencing advancement of the country, or is it the reverse that advanced countries can afford good networks. My belief is it goes both ways. Also one needs to extend the analysis beyond just Africa else it’s kind of a repeat of Pinging Africa , R. Les Cottrell, IEEE Spectrum February 2013. Also see A Simple Tool for Measuring Digital Development , by R. Les Cottrell, IEEE Spectrum February 2013. This is derived from SLAC-PUB-15333.
· Also it may be related to Adib’s paper on Internet Performance and its Reflection on Socioeconomic Development in the ASEAN Countries that was submitted to IEEE and rejected.
Currently, no data is available on PingER on Android due to unavailability of the live IP address. No update 4/19/2017, 7/6/2017. Email sent to Sara Masood. No update 9/24/2017. Any update 10/24/2017.
| Host | State | last seen | Status |
|---|---|---|---|
| pinger.vu.edu.pk | Unable to gather data missing XML/Simple.pm, email sent 6/28/2017, no response. | May 22nd | |
| pinger.uum.edu.my | Unable to gather data since July 17 2017, also does not ping by IP address, by name gives Name does not resolve. Sent Email 8/9/2017. Name does not resolve, address resolves to is.grid.uum.edu.my | July 2017 | Does not ping |
| pinger.ictp.it | Pings but nothing else. Emails sent June 6, 2017, June 28, 2017, August 9, 2017. | Mar 9th | Down |
| www.hepgrid.uerj.br | Due to a major issue with the cluster's electrical infrastructure we will have to shut down all machines, in order to preserve our hardware. The cause of the outage is still being investigated. Emails 11/21/2016, 8/9/2017. Given up. Disabled. | Sept 31, 2016 | Does not ping |
Next meeting: Thursday December 7th 9pm Pacific time, Friday December 8th , 2016 9:00am Pakistan time, Friday December 8th 2017 12:00noon Malaysian and Guangzhou .time
"BIND: An Indexing Strategy for Big Data Processing" that uses PingER data. Submitted and accepted by the 2017 IEEE Region 10 Conference (TENCON) that takes place in November. In Penang Malaysia
The paper title: "Detecting Anomalies from End-to-end Internet Performance Measurements (PingER) using Cluster Based Local Outlier Factor" is submitted in ISPA 2017 (http://trust.gzhu.edu.cn/conference/ISPA2017/). It has been accepted as of 9/17/2017.
The thesis of Aqsa Hameed title “Applying Data Mining and Visualization Techniques on Pinger Data” is published in ODBMs.org and is accessible through http://www.odbms.org/2017/07/applying-data-mining-and-visualization-techniques-on-pinger-data/
Preparing a paper on the impact of the cyclone Verdha that hit the Indian coast along with many countries like Thailand, Sri Lanka, Malaysia, Maldives on December 6th. They use K-Means clustering (see https://en.wikipedia.org/wiki/K-means_clustering) to identify anomalies in packets received (inverse of loss) and maximum RTT. Note that for December 7th the reduction in packets received.
From: Aayush Jain <aayush.2896@gmail.com>
Sent: 24 March 2017 12:31
To: A. Sai Sabitha; harysinha@gmail.com
Subject: PingER Android Team
Progress Made So Far
So far Shivnarayan Rajappa and Rohan Sampson’s team have succeeded in making a bare-bones Android Application that can ping beacons, parse data, and generate a text file in a format specified by SLAC ready for uploading. The proposed model involved the application pulling the beacon list from SLAC’s servers for pinging. However, the present application has a small percentage of the beacon list hard-coded into the application. As of now, the link between the application and proxy server has not been established.
Future Plans
The new team members are:
1. Rohit Raj
2. Shagun Seth
3. Savy Gupta
4. Aayush Jain
5. Tanuj Saraf
Owing to the advancements in Android technologies during the time of development of the project, our team believes that we can create a more capable and robust application for this project. This involves rewriting the entire application from scratch.
We also propose to create a proxy server that can act as an intermediary between the Android application and SLAC’s servers. The proxy server would thus allow handling multiple hosts for greater data collection.
Approach
Our team plans to start off by completing the work on the Android app within 20 days. We will recreate the entire app, with an improved workflow for greater stability. The app will parse the beacon list from SLAC’s servers and save as an XML on the device. The data generated after every ping will be appended to a file after cleaning it up with RegEx matches. We first plan to test the app with only a few members of the ping list (which will later be expanded to auto-update in its entirety).
Once we accomplish our work with the app, we will move on to the task of establishing a proxy server. Our entire team will focus on the components of networking, host management, host authentication, file synchronization, and security.
By the end of the project, the server will be able to handle multiple hosts which would all forward it data, and it would in turn reorganise it again for SLAC’s servers to pull.
The students successfully presented the paper on the PingER implementation on Android.at the confluence 2017 conference.. The paper is submitted to IEEE section.
Tropical cyclone Vardah hit Chennai in India on the Dec 13th. It impacted the Internet, in particular one of Airtel's undersea cables. Les sent email to A. Sai Sabitha to see if PingER from Amity could see any effect.
During the next 6 months their research will study the impact Vardha cyclone that hit the Indian coast(South India/Chennai) and a few other neighboring countries in December 2016 as seen bu PingER.
The idea is to study and analyze the PingER data during the corresponding time frame and deduce significant trends and patterns from the data using
1. Clustering techniques
2. Time series
3. Correlation and Regression concepts
They have fixed the final sequence number change by using regex, and pushed these changes to github repository.
They have installed apache tomcat in the server and plan to use a java file on the server which would connect to the phones that send the request. This java file will then take the input stream received from the phone and write the output stream to a file that would be stored on the server. We are facing some problems regarding a blocked port that is not allowing the phone to connect to the server we are currently working on resolving the issue.
SLAC can then regularly pull these files which would be stored based on the month they are received.
The Android students have started writing a paper on " implementation of pinger on android " .
Next steps:
Extend the target list by getting the Beacon list from SLAC. It is at http://www-iepm.slac.stanford.edu/pinger/pinger.xml on a regular basis and updating the <BeaconList> section at their site. This was part of pinger2.pl.
To a large extent it depends on how we plan to use this.
A next step is to get the data from the phone MA to the archive at SLAC. The current method ping_data.pl requires a public IP address for the phone which may not exist if its is mobile. Getting the MA to put the data to the archive may raise some security issue for the archiver.
Need your feedback on the idea of Measuring the Digital Development of the Countries using PingER data