Versions Compared

Old Version 7

changes.mady.by.user Les Cottrell

Saved on

compared with

New Version 8

changes.mady.by.user Les Cottrell

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  • We are unable to gather data from the Amity MA, or even ping it, emails sent May 28th and June 3rd, 2018, June 30.
  • The Android version of the PingER MA 
    • email sent June 5th and again June 30th:
      • Now that you have support for regularly updating the Beacons, one of the next things is to think about is how to get the data to the repository/archive (SLAC).
        A possibility is outlined in https://confluence.slac.stanford.edu/display/IEPM/Proxy+support+for+PingER.  Perhaps your team might be interested in pursuing step 1.  If necessary we could also think about providing an account at SLAC to enable a student to work on step 2.
      • The students have gone on the summer break, they will be back by the 2nd week in July.

Bebo 

 

Thailand (Updated 6/3/2018)

For his IPv6 monitoring Charnsak is using ~ 100 IPv6 targets from Saqib. They are in the <HostList> of the Ubra PingER MA. It can be seen from http://www-wanmon.slac.stanford.edu/cgi-wrap/pingtable.pl?file=average_rtt&by=by-node&size=100&tick=monthly&from=TH.UBRU.CS.AC.PINGER6&to=WORLD&ex=none&only=all&ipv=all&dataset=hep&percentage=any. It is still running smoothly. However, we are only seeing about 16 nodes in the report.  There may be a bug  bug somewhere or maybe the hosts need to be entered into NODEDETAILS meta database of hosts.

Charnsak is looking at a host in Chan Parsa province in Laos as a potential site for a PingER MA. Charnsak just got approved to make contact with the Champasak University. He expects to set up the MA in the next 4-5 months (say towards end 2018). It also depends on the partner university, and there may be a lot of paperwork.

UUM (Updated 6/7/2018)

...

NUST: (Updated 5/3/2018))

...

No intern has joined Wajahat's Lab. So there is not much progress.

Wajahat proposes to get a list of the new Universities in Pakistan and contact them encouraging them to participate in PingER and set up MA. They have made a list of new university sites, communications networks, Labs in different regions of Pakistan (especially the remote regions) and will make contact.

  • The list of new universities is ready. Just need resources to engage them. 
  • Unsure how this is affected by lack of interns.

Discussion

items

item:

Wajahat is hoping to get few students to work on Master thesis related to pinger data. If possible kindly share ideas related to data analytics which he can share with students

  • Suggestions anyone?

 Wajahat also asked last time:

  •  “Has any work gone into to predicting the cause of failures(blackout, flood, coup (Turkey)) using pinger data?”
  • Les> there have been several case studies looking at the impacts of failures, however nothing on predicting failures. I am not sure how one might use PinGER to predict the cause of a failure. For some cases (e.g. earthquakes, tsunamis, flood, coup), it is unclear to me how Network performance monitoring could add to other means of predicting the cause.
  • Wajahat's thinking is along the lines of "Regarding other sources, I was thinking, internet is the pulse of digital world. Other sources require additional setup that might not be a possibility in developing countries. Internet being a necessity and having other uses is still prevalent in these developing countries. Can it be used as a real time sensor."
  • Again I am forwarding the question in case others in the team may have some suggestions?  Below are some responses:
    • Umar mentioned the use of dark fibre testing (e.g strain) and monitoring for earthquakes. There was a recent paper.
    • The idea of raising alerts from anomalies gives rise to the need for 3 types of alerts: Informational; need action but not critical; severe, wake person up etc. 
    • Things to look at include changes in a metric (e.g. minimum RTT), route changes, loss of connectivity. Correlation with multiple metrics in particular significant route changes and changes in RTT. A requirement is to get few false positives, and false negatives and then triage. Also apply some hysteresis so do not get multiple alerts for the same event.
 
SLAC was unable to gather data from:
  • Umar was thinking that using the PingER data to detect and log anomalies and create alerts might be interesting.  This would include triaging the logs into 3 tiers of alerts:
    • Low = Information; 
    • Medium = needs action but non-critical, create a ticket to be looked at later; 
    • Severe: e.g. wake someone up.
  •  It requires hysteresis so one does not get continuous alerts for say something that has gone down. 
  • This makes a lot of sense for say routers and switches which are usually very critical. 
  • For PingER it might include correlation with feeds from elsewhere (e.g. earthquakes, tsunamis, social unrest)
  • The PingER measurement script (pinger2.pl) has a lot of logs that are prioritized by severity (e.g. target not responding for various reasons for the last n runs of pinger2.pl. 
  • Possibly mine the data with say Splunk.
  • Might be an interesting project for a student.
  • Any further thoughts this month?
 
SLAC was unable to gather data from:
  • 121.52.146.180 (kohat.edu.pk) down since Nov 22/2017. Wajahat recommends continuing at least until the new student is up to speed (3/8/2018). 121.52.146.180 (kohat.edu.pk) down since Nov 22/2017. Wajahat recommends continuing at least until the new student is up to speed (3/8/2018). No data available 3/24/2018.
  • cae.seecs.edu.pk last time we were able to gather any data was February 27th.
  • pinger-ncp.ncp.edu.pk pings but can't gather data 8/11/2017 and 9/16/2017. Contacted. Pings but can't gather data 10/24/2017. They are in the process of restoring 1/17/2018. Still down February 28, 2018, await new student. (3/8/2018). No data 3/24/2018.
  • pinger.isra.edu.pk unable to gather data since 3/6/2018, also does not ping.
  • Wajahat says they will get these nodes up. These have been good nodes. They just need the weekly push. NUST will push them soon.

...

HostStatelast seenStatus
pingersonar-um.myren.net.myNo response6/26/2018Pings
pinger-ncp.ncp.edu.pkpinger-ncp.ncp.edu.pk downNov 29, 2017 
121.56.146.180 (pinger.kohat.edu.pk) DownNov 22nd, 2017 
cae.seecs.edu.pkDownFeb 27, 2018 
pinger.isra.edu.pkDownMarch 6, 2018 
pingeramity.inIt was partially working. Now working on missing beacons.txt file and missing data (i.e data disappears a few hours after it is measured and saved at MA). Also it is unable to access http://www-iepm.slac.stanford.edu/pinger/beacons.txt. Awaits Amity.April 27, 2018 

Bebo 

Discussion

Bebo sent email:

More and more I am convinced that we could identify an interesting and relevant project taking advantage of our PingER experience and involving Blockchain technology.

For example,

https://bitcoin.stackexchange.com/questions/66260/finding-ip-addresses-of-all-bitcoin-miners <https://bitcoin.stackexchange.com/questions/66260/finding-ip-addresses-of-all-bitcoin-miners>

I also think that it may be possible for us to get funding/grant money for such a project which might clearly draw the attention of Blockchain entrepreneurs, conferences, and publications. It might also allow us to attract the collaboration of other institutions and/or universities that might otherwise have not been interested in the basic PingER goals and technology.

Thoughts/responses?

Next Meeting

Next meeting:  Thursday, August 9th 9 pm Pacific time; Friday, August 10th, 2018 9:00 am Pakistan time; 12:00 noon Malaysian & Guangzhou time; and 11 am Thailand time.

...

  1. They are using the native java tools, they are not running the pinger2.pl <http://pinger2.pl>  script on android since the native java tools have the following advantages
    1. easier for user, 
    2. no need for prior installation of any software, e.g. load perl interpreter which may require missing skills, especially for a non technical user
    3. doesn't need a rooted phone
    4. only the apk needs to be installed to run

  2.  They have fixed the final sequence number change by using regex, and  pushed these changes to github repository.

  3. They have installed apache tomcat in the server and plan to use a java file on the server which would connect to the phones that send the request. This java file will then take the input stream received from the phone and write the output stream to a file that would be stored on the server. We are facing some problems regarding a blocked port that is not allowing the phone to connect to the server we are currently working on resolving the issue.

  4.  SLAC can then regularly pull these files which would be stored based on the month they are received. 

  5. The Android students have started writing a paper on " implementation  of pinger  on android " .

  6. Next steps:

    1. Extend the target list by getting the Beacon list from SLAC. It is at http://www-iepm.slac.stanford.edu/pinger/pinger.xml on a regular basis and updating the <BeaconList> section at their site. This was part of pinger2.pl. 

    2. Also they will need a utility to clean out old recorded data (say older than 3 months), since it will be gathered from SLAC (via the proxy) and eventually they may run out memory on the Android.
Discussion

To a large extent it depends on how we plan to use this.

...