Versions Compared

Old Version 8

changes.mady.by.user Les Cottrell

Saved on

compared with

New Version 9

changes.mady.by.user Les Cottrell

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Latest RPi0 for $5. Spec 2*good of RPi2

UTM

Johari has had no response from UTM so far. Not sure whether they still want to be in the project. Last straw, need to contact someone from the network group so that they understand and appreciate the Pinger project. Update 1/6/2016.

Saqib please contact Dr Md Asri Ngadi. Will provide update after talk.

Johari will try and contact Hafizi

Saqib reports:

"Faculty management in the Faculty of Computing-UTM is changed. They ask me to re-locate the PingER server from the faculty server room as students are not allowed for physical access of the server room. Only faculty members are allowed to physically manage the servers in the server room.  Therefore, I asked my supervisor (Dr Md Asri Ngadi/ Prof Hanan) from UTM to take  the responsibility of the server. He is agreed, however, I have to manage the server remotely. Further, he needs an official invitation from your side to fulfill the official requirements. Further, he wants to make this project as a source of an international collaboration and linkages between UTM and SLAC-Stanford."

Les sent back:

The UTM PingER server is part of a worldwide collaboration led by SLAC that includes about 70 sites in over 20 countries. These sites  are monitoring on-going Internet end-to-end performance , both currently and in the past with a history of 18 years of collected data.

SLAC is happy to host postgraduate students from UTM  for a period of up to a year to contribute to this project as visiting scientists.

The students participate in collaborative research and development on the Internet End-to-end Performance Measurement (IEPM) project to monitor, analyze and present information on the end-to-end performance of Internet links. This project includes researching and developing techniques to optimize the measurements for the purposes of characterizing the “Digital Divide”. Dr. Les. Cottrell of SLAC’s Office of the Chief Information Officer is the  host.  All  support for the visit is provided by sources other than SLAC. 

The students are expected to comply with all policies and procedures, including safety guidelines.  Your mentor will provide them with any specific policies, procedures or safety guidelines for their assigned area. As a program participant, the student does not  enter an employee/employer relationship with DOE, SLAC or any other federal agency.

MYREN

No update 8/12/2015, no Update 9/2/2015. 

perfsonar-unimas.myren.net.my is back up and running

We have been unable to gather data from pingersonar-utm.myren.net.my  since September 11th, 2015 pingersonar-utm.myren.net.my. It is pingable  however there is no response from http://pingersonar-utm.myren.net.my/cgi-bin/traceroute.pl?function=ping&target=www-wanmon.slac.stanford.edu&options=-i%200.2

NUST

Hassaan reports: "There are total of 48 active nodes for Pakistan out of which 18 nodes are working properly whereas 4 nodes have configuration issues. Lahore college for Woman has changed its data center and they want to know what do have to do with the new one. Faisalabad PERN had some critical issues and burnt their power supply so they also wanted to know what can be done to overcome that damage. FAST University Karachi and Air University, Islamabad are not in a mood to help as they say that they will put the node up, but haven’t done that in past 2 months [Hassaan would like to discuss this with Dr. Anjum for excluding them from the list]. Last but not the least Quetta HEC regional center wanted to change their static IP to 121.52.157.148."

namal.seecs.edu.pk is working once again

No responses for over 30 days from

PingER at SLAC

Put together a web page on the Fertility Rate and PingER measured Internet performance. The concern is the high Fertility rate in Sub-Saharan which is dominating the global growth in population and could result in over 6B people in Africa and 11B people on the planet by 2100. This puts a huge strain on resources. To reduce the Fertility Rate one needs education and information which is greatly enabled by the Internet. Unfortunately Internet performance is generally at its worst in West, Middle and East Africa where the Fertility Rates are highest. 

Also put together a web page on ways of providing Internet access in hard to reach places, e.g. weather balloons, drones, low earth satellites.

The 2016  annual report on network monitoring is almost ready, it includes a section on PingER.

 Working on the following hosts to be able to gather data

HostStatelast seenStatus
web.hepgrid.uerj.eduemails 5/1/2016, still no response 1/30/2016, email 1/30/2016Dec, 2014does not ping electrical problems
pinger.unesp.bremail 1/30/2016, have made contact they are working on it.over a year agoDoes not ping
pinger.arn.dzemail 1/30/2016Nov 2015Does not ping
pinger.sesame.joemail 1/6/2016. Fixed 1/30/2016 Does not ping, server works
Pinger.stanford.eduemail 1/6/2016May 2015does not ping

Gathering data from the following Pakistani monitoring hosts that we (pinger.slac.stanford.edu) have been unable to gather data from for 6 months or more has been disabled:

 Are there any of these that are likely to return to service?

Last month we Disabled the following (not responding to ping and no data for 12 months)

Next Meeting

Next meeting:  Wednesday Feb 3 8:00pm Pacific Standard Time, Thursday Feb 4, 2016  9:00am Pakistan time, Thursday Feb 4 2016 12:00noon Malaysian time, Thursday Feb 4 2016 02:00am Rio Standard Time.  

...

 

 

Saqib's old supervisor Dr. Md Asri is agreed to appoint a master student to take of PingER in UTM.

 

MYREN

No update 8/12/2015, no Update 9/2/2015. No update Jan 2016. Should we try and revive?  The hosts all look fine.

 

UOA (Saqib)

Name: Aqsa Hameed

Title: visualization of PingER historical data using warehouse

Status: Our idea is to develop a warehouse in our university and make it publicly available.

 

Name: Sara Masood

Title: PingER Internet Performance Monitoring Agent On Android Device

Status: She sent you & Dr Bebo the mockup screens of the proposed Android Monitoring Agent. She is waiting for your review. Further, she requires web services to get and save PingER data.

 

Name: Saba Muzamil

Title: She is interested to work on Environment that supports the execution of analytical distributed queries in Hadoop Cluster.

 

Name: Tahseen

Title: Handling Missing Data in PingER by using Artificial Intelligence

Status: He is doing the literature review to find out the best algorithm to handle missing data in PingER.

 

About IoT:

Title: Air Quality measuring network in University of Agriculture Faisalabad.

Status: Now we have two Raspberry Pi-2 and few air quality measuring sensor i.e.; NO, CO, O3. Currently, working on the configuration and programming of the Pi-2 and the sensors.

NUST

Hassaan reports: "There are total of 48 active nodes for Pakistan out of which 18 nodes are working properly whereas 4 nodes have configuration issues. Lahore college for Woman has changed its data center and they want to know what do have to do with the new one. Faisalabad PERN had some critical issues and burnt their power supply so they also wanted to know what can be done to overcome that damage. FAST University Karachi and Air University, Islamabad are not in a mood to help as they say that they will put the node up, but haven’t done that in past 2 months [Hassaan would like to discuss this with Dr. Anjum for excluding them from the list]. Last but not the least Quetta HEC regional center wanted to change their static IP to 121.52.157.148."

namal.seecs.edu.pk is working once again

No responses for over 30 days from

PingER at SLAC

Put together a web page on the Fertility Rate and PingER measured Internet performance. The concern is the high Fertility rate in Sub-Saharan which is dominating the global growth in population and could result in over 6B people in Africa and 11B people on the planet by 2100. This puts a huge strain on resources. To reduce the Fertility Rate one needs education and information which is greatly enabled by the Internet. Unfortunately Internet performance is generally at its worst in West, Middle and East Africa where the Fertility Rates are highest. 

Also put together a web page on ways of providing Internet access in hard to reach places, e.g. weather balloons, drones, low earth satellites.

The 2016  annual report on network monitoring is almost ready, it includes a section on PingER.

 Working on the following hosts to be able to gather data

HostStatelast seenStatus
web.hepgrid.uerj.eduemails 5/1/2016, still no response 1/30/2016, email 1/30/2016Dec, 2014does not ping electrical problems
pinger.unesp.bremail 1/30/2016, have made contact they are working on it.over a year agoDoes not ping
pinger.arn.dzemail 1/30/2016Nov 2015Does not ping
pinger.sesame.joemail 1/6/2016. Fixed 1/30/2016 Does not ping, server works
Pinger.stanford.eduemail 1/6/2016May 2015does not ping

Gathering data from the following Pakistani monitoring hosts that we (pinger.slac.stanford.edu) have been unable to gather data from for 6 months or more has been disabled:

 Are there any of these that are likely to return to service?

Last month we Disabled the following (not responding to ping and no data for 12 months)

Next Meeting

Next meeting:  Wednesday Feb 3 8:00pm Pacific Standard Time, Thursday Feb 4, 2016  9:00am Pakistan time, Thursday Feb 4 2016 12:00noon Malaysian time, Thursday Feb 4 2016 02:00am Rio Standard Time.  

Old Items

UOA (Saqib) placed here 2/3/2016.

 

  • Saqib has a  5  MS students from the Database team

    • Sara Massoud came up with ideas to improve the info put together by Amity

    • Sabah Massoumil working on Linked Open Data (please excuse spelling)

    • Aqsa Hameed working on big data/analysis of PingER data. The initial idea was to set up a big data/big analysis 

    • Aqsa Hameed has been working with Anjum to look at a project to create a Hadoop/Cloudera PingER warehouse to enable easier, more powerful access to PingER historical data. 

      • Thinking a bit on Aqsa her work appears to be related to the work done by Thiago on PingER warehouse using a cluster/Cloudera/HDFS/Impala earlier this year. In particular see the presentation at NETAPPS2015 (see https://confluence.slac.stanford.edu/download/attachments/123309267/NETAPPS_PRESENTATIONrevLuiza.pptx. Once the paper is published we can also provide her with that (Adib will let us know when this is OK). 

      •  Les emailed the relevant people to put them in contact with one another. It appears there is a lot of overlap between what Aqsa proposed as a Masters project and what Thiago has already done. However Thiago's system was mainly a proof of concept and not in production. We need to look at the next steps: Internet access, auto-updating of information in near realtime, production service, maintainable etc. With this in place one could really mine the data looking for all kinds of interesting correlations, clustering, event impacts, comparisons etc.

        • Given that such a warehouse is available, then the next step would be to automatically create queries that would produce in near real-time the plots we produce manually for the PingER annual reports.

        • Following this publish the data in RDF to tie in with the Southampton RDF web observatory repository (similar to what Renan did as a proof of concept).

        • Since Thiago's warehouse is only available at SLAC,  it may assist to get Aqsa an account at SLAC, alternatively Aqsa will need to set up a repository at here home base.

        • There is some documentation written by Les on the usage of the warehouse at SLAC. It is at: PingER Data Warehouse using Big Data with Cloudera on Nebula.

        • Les will contact Thiago to see if there is other documentation or where to find the programs etc.

        • We need to put together all we know:

      • Tehseen is working on missing PingER data.

  • Saqib has submitted a project with title "A Fundamental Active Internet Performance Monitoring Framework for Pakistan Education & Research Network (PERN) in University of Agriculture, Faisalabad"

    • Project is accepted.

    • working on a project to develop a node using Raspberry Pi 2 and IoT to measure the air and soil quality.

      • Have 2 RPis and setting up air quality with sensors from market, idea is to distribute on campus.

    • Currently, students are analyzing the project to develop a problem statement for their research project.

  • For the GeoLocation Saqib should contact Anjum

  • Project on Internet performance has been accepted, but no funding yet.

Others

  • Jan 5, 2015 Hassaan reports "I have received revisions on my proposal and these days I am revising my proposal. In the meanwhile, I have also added another student (Anas Abrar) on this project. He is in learning phase and will follow the nodes which are not working. I shall give you an update very soon. "

    •  Hassaan is  very hopeful that if the proposal is accepted then we can easily have a full time RA for the project.

    Hassaan has re-submitted the proposal after revisions. He would like to get Anas Abrar more trained on monitoring operation and then will inform us to add him to the mailing list at http://pinger.unimas.my/pinger/contact.php.

     

  • Oct 2015. Following the last meeting, Anjum, Hassaan and Les met to discuss the way forward. 

    • "Adnan currently is unable to find resources for handling the project. Similarly, there is no progress on hiring of a full time RA by NUST HQ. 
    • However, I (Hassaan) checked from HEC about the proposal that I submitted last year. They have informed me that 2 reviewers have asked for revisions while they are waiting for the third review. I am very hopefull about it. If the proposal is accepted then we can easily have a full time RA for the project. I have plans to talk to Dr. Zaidi about hiring an RA on assuming that our proposal will be accepted by HEC. We can then get his salary deducted later from the HEC project. I shall update you very soon in this regard." 

    • Hassaan is  waiting to hear from HEC about the comments on the proposal. 

    • Moreover, he has asked a student to work on the project for the time being. His name is Mian Anas however he will need few weeks to understand the project. 

  • Thiago completed setting up the  PingER data SQL Impala warehouse running on a Nebula/Cloudera cluster using the Hadoop File System (HDFS). Unfortunately it is not currently accessible from outside SLAC. There have been several attempts to provide outside access, but no success yet, we need to engage the subject matter experts. Thiago is now a SLAC associate so he still has an account at SLAC. There was a cyber security alert on the version of java installed with Cloudera. Les has replaced the cloudera version of java which should fix the vulnerability. However the new version has not been tested.

...