...
Hassaan Khaliq, Kashif, Raja, Samad Riaz (SEECS); Saqib+, Aqsa (UOA); Johari+, Nara, Adnan Khan+ (UNIMAS); Abdullah, Badrul, Anjum, Ridzuan, Ibrahim (UM); Hanan (UTM); Adib+, Fatima (UUM); Fizi Jalil (MYREN); Thiago, Les+, Bebo (SLAC).
...
- Responded but Unable to attend:
? Individual emails sent
Actual attendees:
Saqib (but lost connection), Johari, Adnan, Adib, Les
Administration
- Membership of pinger-my in https://groups.google.com
Huffington Post article on PingER and African Internet performance, see http://www.huffingtonpost.com/david-tereshchuk/a-giant-leap-in-2016-africa_b_8901556.html
Anjum said it would be good for Hassaan to attach this to his proposal. Any action?
Les is working with the SLAC CIO to try and put something together for publicity concerning the life of PingER.
- Johari thinking of a PingER workshop later this year.
- NETAPPS2015 1-3 December 2015 was very successful, five papers were presented in the PingER session
- Currently, NetApps organizers are in the process of sending NETAPPS2015 proceedings to be indexed in the major indexing services. - Adib any update
- Papers are now SLAC-PUBS
Survey on Big Data Indexing strategies, also SLAC-PUB-16460, Fatima Bintu Adama, Adib Habbal, Suhaidi Hassan, R. Les Cottrell. Bebo White, Ibrahim Abdullahi, , submitted to NETAPPS 2015.
Worldwide Internet Performance Measurements Using Lightweight Measurement Platforms, alsoSLAC-PUB-16461,R. Les Cottrell, Thiago Barbosa, Bebo White, Johari Abdullah and Topher White, submitted to NETAPPS 2015.
- Applying Data Warehousing and Big Data Techniques to Analyze Internet Performance, also SLAC-PUB-16464, T. M. S. Barbosa, R. F. Souza, S. M. S. Cruz, M. L. Campos and R. L. Cottrell, submitted to NETAPPS 2015.
PingER Malaysia-Internet Performance Measuring Project: A Case Study, also see SLAC-PUB-16462, Saqib Ali, R. Les Cottrell, Anjum Naveed, submitted to NETAPPS 2015.
Adaptive Geolocation of Internet Hosts, also SLAC-PUB-16463, Raja A.A. Khan, Roger L.A. Cottrell, Anjum Naveed, SLAC-PUB-16360, submitted to NETAPPS 2015.
- Good news from NETAPPS TPC that some of our Pinger team papers are recommended for possible publication in journal JICT (Scopus Index). They will contact us shortly to provide further detail on the journal requirements.
- Papers are now SLAC-PUBS
9 students from Amity University near New Delhi in India interested in coming to 9 students from Amity University near New Delhi in India interested in coming to SLAC. Bebo sent reminder recently, response below:
They have gone through the installation instructions for traceroute.pl, ping_data.pl and pinger2.pl. pinger2.pl appears to work, however we cannot access eth data by ping_data.pl. There is also a problem that their web server appears to be local, see Setting up PingER servers at Amity
- Joao a Brazillian student identified by Renan is interested in a 3 month internship at SLAC similar to those completed by Renan and Thiago.
- We just need one more signature then I believe we can send the invitation
...
- slac-pinger/pinger created by topherwhite. PingER project https://github.com/slac-pinger/pinger
- Now we have it we can share with Amity to check it works.
- Bebo reports that someone from Amity requested to be a member of the Android group.
...
UUM
- Saqib has a 5 MS students from the Database team
Sara Massoud came up with ideas to improve the info put together by Amity
Sabah Massoumil working on Linked Open Data (please excuse spelling)
Aqsa Hameed working on big data/analysis of PingER data. The initial idea was to set up a big data/big analysis
Aqsa Hameed has been working with Anjum to look at a project to create a Hadoop/Cloudera PingER warehouse to enable easier, more powerful access to PingER historical data.
Thinking a bit on Aqsa her work appears to be related to the work done by Thiago on PingER warehouse using a cluster/Cloudera/HDFS/Impala earlier this year. In particular see the presentation at NETAPPS2015 (see https://confluence.slac.stanford.edu/download/attachments/123309267/NETAPPS_PRESENTATIONrevLuiza.pptx. Once the paper is published we can also provide her with that (Adib will let us know when this is OK).
Les emailed the relevant people to put them in contact with one another. It appears there is a lot of overlap between what Aqsa proposed as a Masters project and what Thiago has already done. However Thiago's system was mainly a proof of concept and not in production. We need to look at the next steps: Internet access, auto-updating of information in near realtime, production service, maintainable etc. With this in place one could really mine the data looking for all kinds of interesting correlations, clustering, event impacts, comparisons etc.
Given that such a warehouse is available, then the next step would be to automatically create queries that would produce in near real-time the plots we produce manually for the PingER annual reports.
Following this publish the data in RDF to tie in with the Southampton RDF web observatory repository (similar to what Renan did as a proof of concept).
Since Thiago's warehouse is only available at SLAC, it may assist to get Aqsa an account at SLAC, alternatively Aqsa will need to set up a repository at here home base.
There is some documentation written by Les on the usage of the warehouse at SLAC. It is at: PingER Data Warehouse using Big Data with Cloudera on Nebula.
Les will contact Thiago to see if there is other documentation or where to find the programs etc.
We need to put together all we know:
Paper from Thiago on PingER warehouse presented in Malaysia at NETAPP2015 Thiago M. Da S. Barbosa, Renan F. Souza, Sergio M. S. Da Cruz, Maria L. Campos and R. Les Cottrell. Applying Data Warehousing and Big Data Techniques to Analyze Internet Performance
Document on Warehouse
Documents on PingERLOD
Linked Open Data Publication Strategies: Application in Networking PPerformance Measurement Data, Renan Souza, Les Cottrell, Bebo White, Maria Campos, Marta Mattoso, poster presented at the BigDataScience - Stanford conference, CA, USA May 27-31, 2014.
Reviews/proposals/ from Aqsa/Fatima
Survey on Big Data Indexing strategies, Fatima Bintu Adama, Adib Habbal, Suhaidi Hassan, R. Les Cottrell. Bebo White, Ibrahim Abdullahi.
Saqib will provide Aqsa's
Tehseen is working on missing PingER data.
- Saqib has submitted a project with title "A Fundamental Active Internet Performance Monitoring Framework for Pakistan Education & Research Network (PERN) in University of Agriculture, Faisalabad"
Project is accepted.
working on a project to develop a node using Raspberry Pi 2 and IoT to measure the air and soil quality.
Have 2 RPis and setting up air quality with sensors from market, idea is to distribute on campus.
Currently, students are analyzing the project to develop a problem statement for their research project.
For the GeoLocation Saqib should contact Anjum
- Project on Internet performance has been accepted, but no funding yet.
UUM
Fatima presented her work at NetApps. Fatima has passed her master oral examination (Viva) with minor correction and she is submitting her final dissertation coming Sunday to UUM school of computing. Indeed, we are very grateful to PingER team members support and help. Currently, Fatima is planning to pursue her PhD and possibly on a topic related to her master dissertation. We need PingER team suggestions and advice before we can decide on the topic. Adib will attend next meeting to further discuss on this.
UM
Ibrahim has extracted the PingER Zip manually. He is reconstructing 11Gb of data, has 15GBytes of data there. He is trying to use SPAR to classify the data. He also was looking at RDF. the next step is to use MapReduce to organize and reduce the output of data so can visualize it. He will be using a the same techniques he used for looking at 1996-2006 weather data. However at the moment he cannot access his VMs from myren for the last two weeks, even for the myren site, He has emailed them but till now they have not fixed the issue. He has updated all his data in their cloud. No update 1/6/2016
UNIMAS
UTM
Saqib's old supervisor Dr. Md Asri is agreed to appoint a master student to take of PingER in UTM.
MYREN
No update 8/12/2015, no Update 9/2/2015. No update Jan 2016. Should we try and revive? The hosts all look fine. Johari will send email to Fitzi. Adib has a meeting with MYREN will see if he can make cotact
UOA (Saqib)
Name: Aqsa Hameed
Title: visualization of PingER historical data using warehouse
Status: Our idea is to develop a warehouse in our university and make it publicly available.
Name: Sara Masood
Title: PingER Internet Performance Monitoring Agent On Android Device
Status: She sent you & Dr Bebo the mockup screens of the proposed Android Monitoring Agent. She is waiting for your review. Further, she requires web services to get and save PingER data.
Name: Saba Muzamil
Title: She is interested to work on Environment that supports the execution of analytical distributed queries in Hadoop Cluster.
Name: Tahseen
Title: Handling Missing Data in PingER by using Artificial Intelligence
Status: He is doing the literature review to find out the best algorithm to handle missing data in PingER.
About IoT:
Title: Air Quality measuring network in University of Agriculture Faisalabad.
- Fatimah: she has decided to take one semester off before she pursues her PhD. She is planning to use this semester to explore the area more and finalize her PhD topic. Any suggestions or ideas are welcomed.
- PINGER IPv6: I have one Practicum student interested to work on PingER IPv6. Based on the information provided in PingER site, “(pinger2.pl) will work with both IPv4 and IPv6” while “ping_data.pl will need some mods also.” Following our today’s meeting, the student is expected to create ping6_data.pl which will work on flat file (just to be specific). Dr. Les, has suggested to provide an account for us at SLAC to be able to test. In addition, it would be great if we can get some documentation about ping_data.pl so the student will get clear understanding on how the current system work and this will help him to progress faster.
UNIMAS
UOA (Saqib)
Name: Aqsa Hameed
Title: visualization of PingER historical data using warehouse
Status: Our idea is to develop a warehouse in our university and make it publicly available.
Name: Sara Masood
Title: PingER Internet Performance Monitoring Agent On Android Device
Status: She sent you & Dr Bebo the mockup screens of the proposed Android Monitoring Agent. She is waiting for your review. Further, she requires web services to get and save PingER data.
Name: Saba Muzamil
Title: She is interested to work on Environment that supports the execution of analytical distributed queries in Hadoop Cluster.
Name: Tahseen
Title: Handling Missing Data in PingER by using Artificial Intelligence
Status: He is doing the literature review to find out the best algorithm to handle missing data in PingER.
About IoT:
Title: Air Quality measuring network in University of Agriculture Faisalabad.
Status: Now we have two Raspberry Pi-2 and few air quality measuring sensor i.e.; NO, CO, O3. Currently, working on the configuration and programming of the Pi-2 and the sensors.
Project on Internet performance has been accepted, but no funding yet.
UTM
Saqib's old supervisor Dr. Md Asri is agreed to appoint a master student to take of PingER in UTM.
UM
Ibrahim has extracted the PingER Zip manually. He is reconstructing 11Gb of data, has 15GBytes of data there. He is trying to use SPAR to classify the data. He also was looking at RDF. the next step is to use MapReduce to organize and reduce the output of data so can visualize it. He will be using a the same techniques he used for looking at 1996-2006 weather data. However at the moment he cannot access his VMs from myren for the last two weeks, even for the myren site, He has emailed them but till now they have not fixed the issue. He has updated all his data in their cloud. No update 1/6/2016
MYREN
No update 8/12/2015, no Update 9/2/2015. No update Jan 2016. Should we try and revive? The hosts all look fine. Johari will send email to Fitzi. Adib has a meeting with MYREN will see if he can make cotact
Status: Now we have two Raspberry Pi-2 and few air quality measuring sensor i.e.; NO, CO, O3. Currently, working on the configuration and programming of the Pi-2 and the sensors.
NUST
Hassaan reports: "There are total of 48 active nodes for Pakistan out of which 18 nodes are working properly whereas 4 nodes have configuration issues. Lahore college for Woman has changed its data center and they want to know what do have to do with the new one. Faisalabad PERN had some critical issues and burnt their power supply so they also wanted to know what can be done to overcome that damage. FAST University Karachi and Air University, Islamabad are not in a mood to help as they say that they will put the node up, but haven’t done that in past 2 months [Hassaan would like to discuss this with Dr. Anjum for excluding them from the list]. Last but not the least Quetta HEC regional center wanted to change their static IP to 121.52.157.148."
...