Versions Compared

Old Version 11

changes.mady.by.user Les Cottrell

Saved on

compared with

New Version 12

changes.mady.by.user Les Cottrell

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

? Individual emails sent

Actual Attendees

Saqib, Les, Bebo, Umar

Others

Wajahat is having an Artificial Intelligence workshop at SEECS NUST. He might not be able to attend the meeting.

Administration

...

  • Amity MA is unreliable so using it for a case study does not appear fruitful.
  • The Android version of the PingER MA, is described with  comments at  ePingER on Android Native - Amity project (this a proposal/description from Aayush Jain)
    • It describes a multipurpose, stand-alone device that can be widely distributed, something that we have brainstormed about for a long time. 

    • Bebo mentioned it to Topher and he feels that their app (when completed and vetted by his team) could easily be installed as a default service on his rainforest monitors (certainly future ones, not devices already in place). Merging the service data that he already collects with that unique to PingER has the potential to lead to some interesting results. 

    • Bebo asks: should we try to increase our communication with Amity and do we have faith that they would follow through?

    • How should we proceed? Discussion

Thailand (Updated 7/5/2018)

    •  

      • We agreed to request Amity to share the App and instructions with us; we will look at installing on a jailbroken Android phone at the San Francisco end and try it out.

      • Also encourage them to put together a paper.

      • Also propose a meeting between: Bebo, Umar and Les. and the Amity folks.

Thailand (No update 8/9/2018)

There was a problem with the IPv6 MA which was solved by Charnsak rebooting the There was a problem with the IPv6 MA which was solved by Charnsak rebooting the MA on August 4th.

For his IPv6 monitoring site pinger6.ubru.cs.ac.th Charnsak has a pinger.xml configuration file with over 160 IPv6 targets. However, there is a huge discrepancy since according to pingtable.pl there are only about 13 targets responding. We need to get the latest pinger2.pl measurement agent script installed at Ubru so we can get better logging and see why the other hosts are not being monitored. Les and  Charnsak are working on this.  -   Charnsak plans to remedy next week.

Charnsak is looking at a host in Champasak University, Chan Parsa province in Laos as a potential site for a PingER MA. Charnsak just got approved to make contact with the Champasak University. He expects to set up the MA in the next 4-5 months (say towards end 2018). It also depends on the partner university, and there may be a lot of paperwork.

UNIMAS (

...

No update 8/9/2018)

Need to add Umar Kalim to http://pinger.unimas.my/pinger/contact.php. Johari can't ssh into the server so he will go to it on Monday.  He will also upload the new UNIMAS PingER website next week.

Sent reminder email 8/6/2018.

UUM (

...

No update 8/9/2018)

Les has sent Adib updates to Figs 3, 4, 5 to extend out to 2018. This is for the paper  Socio-economic Development Indices and Their Reflection on Internet Performance in ASEAN Countries

Adib will submit the paper to World Development: https://www.journals.elsevier.com/world-development


NUST: (Updated 7/5/2018)), no update 9/8/2018)

No intern has joined Wajahat's Lab. So there is not much progress. The students are away on summer vacation so no progress.

Wajahat proposes to get a list of the new Universities in Pakistan and contact them encouraging them to participate in PingER and set up MA. They have made a list of new university sites, communications networks, Labs in different regions of Pakistan (especially the remote regions) and will make contact.

  • The list of new universities is ready. Just need resources to engage them. 
  • Unsure how this is affected by lack of interns.

Discussion item

:

(7/5/2018)

Wajahat

Wajahat is hoping to get few students to work on Master thesis related to pinger data. If possible kindly share ideas related to data analytics which he can share with students. 

 Wajahat also asked last time:

  •  “Has any work gone into to predicting the cause of failures(blackout, flood, coup (Turkey)) using pinger data?”
  • Les> there have been several case studies looking at the impacts of failures, however nothing on predicting failures. I am not sure how one might use PinGER to predict the cause of a failure. For some cases (e.g. earthquakes, tsunamis, flood, coup), it is unclear to me how Network performance monitoring could add to other means of predicting the cause.
  • Wajahat's thinking is along the lines of "Regarding other sources, I was thinking, internet is the pulse of digital world. Other sources require additional setup that might not be a possibility in developing countries. Internet being a necessity and having other uses is still prevalent in these developing countries. Can it be used as a real time sensor."
  • Again I am forwarding the question in case others in the team may have some suggestions?  Below are some responses:
    • Umar mentioned the use of darkfibre testing (e.g strain) and monitoring for earthquakes. There was a recent paper.
  • Umar was thinking that using the PingER data to detect and log anomalies and create alerts might be interesting.  This would include triaging the logs into 3 tiers of alerts: It requires hysteresis so one does not get continuous alerts forsay something that has gone down. 
    • Low = Information; 
    • Medium = needs action but non-critical, create a ticket to be looked at later; 
    • Severe: e.g. wake someone up.
  • This makes a lot of sense for say routers and switches which are usually very critical. 
  • The PingER measurement script (pinger2.pl) has a lot of logs that are prioritized by severity (e.g. target not responding for various reasons for the last n runs of pinger2.pl.) 
  • Tiering of events into alert levels would be the focus of the research
  • PingER might be an interesting field to try out the marshaling of events and categorizing into alert levels/tiering. 
  • Possibly mine the data with say Splunk. Might be an interesting project for an MS orPhDstudent.
    • Umar looked at this: Splunk would be useful for rudimentary for simple scenarios, but not for detailed work.
  • For PingER it might include correlation with feeds from elsewhere (e.g. earthquakes, tsunamis, social unrest)
  • Prediction/tracking:
    • For unexpected events such as earthquakes/tsunamis, it is doubtful PingER data would be useful for predictions
    • For situations that are developing such as hurricanes, civil unrest: knowing roughly where it is developing might make it possible to temporarily increase the density of targets in the region. 
      • To do this would mean having a lot more targets available worldwide ready to be monitored.
      • One would need to be careful to ensure that the extra monitoring does not interfere with Internet access in the area.
    • This tracking might be very valuable for less developed areas where there may be few other (non-PingER) measurements available. However, it can be hard to find the coordinates of ping responding targets is such areas. This would need to be one of the tools developed.

Wajahat is having an Artificial Intelligence workshop at SEECS NUST. He might not be able to attend the meeting.Wajahat says: "I was discussing the blockchain related email that you sent earlier with Dr. Taha (security researcher at SEECS NUST). He was excited about it. He wanted to try a few things and had a few queries which I was not able to answer.  He might join us in our future meetings."

...

  • 121.52.146.180 (kohat.edu.pk) down since Nov 22/2017. Wajahat recommends continuing at least until the new student is up to speed (3/8/2018). No data available 3/24/2018.
  • cae.seecs.edu.pk last time we were able to gather any data was February 27th.
  • pinger-ncp.ncp.edu.pk pings but can't gather data 8/11/2017 and 9/16/2017. Contacted. Pings but can't gather data 10/24/2017. They are in the process of restoring 1/17/2018. Still down February 28, 2018, await new student. (3/8/2018). No data 3/24/2018. Still down 8/6/2018.
  • pinger.isra.edu.pk unable to gather data since 3/6/2018, also does not ping.
  • Wajahat says they will get these nodes up. These have been good nodes. They just need the weekly push. NUST will push them soon.  No update 8/9/2018.

UAF/GHZU (Updated

...

78/

...

9/2018)

...

Saqib's future at GZHU will be much clearer after 3 months. His current contract expires February 2019. 

Anomaly detection

  • The paper on anomaly detection was accepted at DataCon 2018 in Greece.

Blockchain

  • Looking into moving PingER to a "blockchain" database good for decentralizing distribution of data. Monitoring sites would then be able to write to a distributed ledger. This would change the architecture to a more peer to peer architecture. It helps with continuity of PingER since reduces dependence on a single site (SLAC). See BlockChain in Future PingER Projects. Bebo sent several references to Saqib who has looked at them. We could start with real-time data without including the whole archive, i.e. in parallel to the continued centrally managed archive. It would be a private Blockchain and hence not be as compute intensive as a public blockchain. 

...

  • There was a meeting to discuss blockchain possibilities, see 20180709 PingER Meeting on Blockchains. 
  • Bebo's impression is that Saqib will lead in putting the ides in his paper into practice. Saqib will need some students. 

...

  • Saqib is OK with this. He has 2 masters students but they are working in different areas.  Maybe NUST can assist with this.Saqib's partner gave a talk/paper on work so far at the New York meeting on July 31st. The talk went fine but there were not many comment/questions.
  • Saqib is pursuing PingER and Blockchain. He is looking at different references shared by Prof. Bebo and the implementation details using Hyperledger Fabric.

...

  • Saqib is looking at making a test implementation. The blocksize will be 2MB-10MB. It does not appear to be computationally expensive. He will start testing with Internet of Thinggs measurements such as humidity and temperature
  • There was a discussion on the use of DataBases and whether they could be avoided by caches. Hyperlogic keys are not in an SQL DB, basically it appears like a cache.  there was a question whether a 10MB block would be adequate for PingER. Foe example PingER from SLAC has about 700+ targets, the measurements are each 30 mins (48/day) and for pings of 100B and 1000 Bytes i.e. 2*48*700 measurements and each measurement is ~ 140Bytes, so a days sets of measurements from just SLAC is ~ 10MBytes. We could choose to ignore the 1000Byte pings which would reduce it to ~ 5MBytes/day. The latency of retrieving a block is proportional to the block size. Things will be clearer after the test set up is in use.

IPv6 measurements

  • There are now several months of IPv6 PingER measurements from GZHU/BJ, UBRU and SLAC. It is time to think about in-depth analysis of the data.
  • Umar has not made much progress on the comparisons of IPv6 vs IPv4 or TCP vs ICMP.  
  • The PingER measurements would provide longer time spans where one might look for changes with time such as diurnal changes, impact of holidays, anomalies etc. 

PingER at SLAC (Updated 6/7/2018, no update 8/9/2018)

Umar looking at extending the comparison IPv6 vs IPv4 ping RTTs and TCP vs ICMP/ping RTTs. 

  • See Towards Analysis of ICMP vs TCP Ping Latencies - Umar
    • Looked into Traffic Differentiation - Rate Limiting vs. Traffic Prioritization (QoS)

    • IPv6 results gathered using ping-vs-tcp.pl script. About 56 nodes with IPv6 addresses, 14 of which responded with Npings

    • IPv4 results gathered from SLAC and Virginia Tech
      • SLAC's batch may be downloaded here (approx. 24 MB)
      • Skimmed results; findings are pretty much the same as before
  • Pending
    • Identified relevant events in the network stack that highlight timing (_RECVFROM, _RECVMSG, _IP_RECV, _NETIF_RX etc.). Looking for instrumentation that enables us to measure timestamps. We also need to figure out how to determine whether ICMP & TCP traffic are treated differently? and then how to measure the difference?
      • perf-tools allows us to measure transport events
      • If we could assume that the path for ICMP & TCP through the network is the same, then the only difference between two (controlled) tests would be the time spent in the transport layers. This can be measured using perftools. 
      • However, such measurements must be made in a controlled environment where ICMP and TCP are treated the same. (I say so because some results — e.g., in East Asia and South Asia — clearly show that ICMP performs much worse than TCP.)
    • We would also need to cater for cross traffic and queuing delays. Given how small the differences are, one may argue that the variations in measurements are due to cross traffic. Perhaps we should start with controlled tests and then see if real world measurements reflect similar behavior.
    • We need to setup a test environment. We can either setup a bare-metal box or use a VM. 
      • I will see if I can arrange for a bare-metal box.

PingER IPV6 support

  • Les identified and sanitized a Cross Site Scripting (XSS) exposure in ping-data.pl. He will send out a notice to please update.
  • Les is cleaning up the Beacons (i.e. removing non-responders and replacing, adding Beacons for countries with no Beacons).
HostStatelast seenStatus
pinger.ascr.doe.govDown since 21st July with a bad disk, contact will move MA to a VM7/20/2018Fixed August 9th, 2018
pingersonar-um.myren.net.myNo response6/26/2018Pings
pinger-ncp.ncp.edu.pkpinger-ncp.ncp.edu.pk downNov 29, 2017Disabled
121.56.146.180 (pinger.kohat.edu.pk) DownNov 22nd, 2017 
cae.seecs.edu.pkDownFeb 27, 2018 
pinger.isra.edu.pkDownMarch 6, 2018 
pingeramity.inIt has been working since 28th July. It is unclear how stable it is.April 27, 2018 

Next Meeting

Next meeting:  Thursday, September 6th 9 pm Pacific time; Friday, September 7th, 2018 9:00 am Pakistan time; 12:00 noon Malaysian & Guangzhou time; and 11 am Thailand time.

...

Old information

Discussion item (7/5/2018)

Saqib sent an email to the team:

In early years of PingER, the framework was designed to check the latency and other Internet performance metrics between CERN and SLAC to facilitate the data transfer between the two sites.

"I am thinking, is there any possibility to use PingER to monitor the health of the Bitcoin blockchain network? Since the latency is critical in Bitcoin blockchain network as all the incentives depend on the propagation of transactions and mined blocks. Thus, I am only interested in measuring the latency to check its effect on the propagation of the transactions and mined block among different mining pools. I think if we can do such thing on a historical basis as PingER already does for the Internet, it will increase the worth of the framework and its usability.

Maybe a few test experiments can guide us to a good research paper. I am not sure about the feasibility idea, therefore, need your kind feedback"

Umar responded:

"This is an interesting idea. I would like to think about it a bit more though before I respond at length.
From what I gather, nodes in the network may appear and disappear without notice. Therefore, it is safe to assume that the network state is changing at all times. I wonder, what is it that we would be measuring, for it to be meaningful. Would it be the latency between full nodes? Would it be the latency from a PingER monitoring node to the full nodes? Would the monitoring nodes be representative of typical clients? Is latency the metric to measure? What are all the projects that measure latency or other metrics? (The bitcoin nodes project is interesting, which shows the size of the network. Similarly, the project that measures average-transaction-completion time is relevant. As you pointed out, the bitcoin stats about data exchange are relevant too.)
Thank you for sharing the URLs. I will think about it a bit more and get back to you."
Bebo emailed:

  • More and more I am convinced that we could identify an interesting and relevant project taking advantage of our PingER experience and involving Blockchain technology.I also think that it may be possible for us to get funding/grant money for such a project which might clearly draw the attention of Blockchain entrepreneurs, conferences, and publications. It might also allow us to attract the collaboration of other institutions and/or universities that might otherwise have not been interested in the basic PingER goals and technology.

Though the transactional latency is important especially for achieving consensus, most of the latency is computational that is much greater than the communications latency. Possibly PingER RTT could be included in a measure of BlockChain health and status.  Bebo sent a reference (https://mastanbtc.github.io/blockchainnotes/consensustypes/) pointing out milliseconds may make a difference to miners to get rewards which is important for BlockChain cash
Saqib sent email of an analysis of the impact of various RTTs, see https://ethereum.stackexchange.com/questions/18201/does-network-latency-significantly-affect-mining-rewards.
A possible problem is finding the IP addresses of the bitcoin miners. Bebo dug up a URL showing how it can be done, it is at https://bitcoin.stackexchange.com/questions/66260/finding-ip-addresses-of-all-bitcoin-miners.

Bebo's impression is that Saqib will lead in putting the ideas in his paper into practice. Saqib will need some students.  Saqib's boss is going to the NY City meeting.

Discussion item

Saqib sent an email to the team:

In early years of PingER, the framework was designed to check the latency and other Internet performance metrics between CERN and SLAC to facilitate the data transfer between the two sites.

"I am thinking, is there any possibility to use PingER to monitor the health of the Bitcoin blockchain network? Since the latency is critical in Bitcoin blockchain network as all the incentives depend on the propagation of transactions and mined blocks. Thus, I am only interested in measuring the latency to check its effect on the propagation of the transactions and mined block among different mining pools. I think if we can do such thing on a historical basis as PingER already does for the Internet, it will increase the worth of the framework and its usability.

Maybe a few test experiments can guide us to a good research paper. I am not sure about the feasibility idea, therefore, need your kind feedback"

Umar responded:

"This is an interesting idea. I would like to think about it a bit more though before I respond at length.
From what I gather, nodes in the network may appear and disappear without notice. Therefore, it is safe to assume that the network state is changing at all times. I wonder, what is it that we would be measuring, for it to be meaningful. Would it be the latency between full nodes? Would it be the latency from a PingER monitoring node to the full nodes? Would the monitoring nodes be representative of typical clients? Is latency the metric to measure? What are all the projects that measure latency or other metrics? (The bitcoin nodes project is interesting, which shows the size of the network. Similarly, the project that measures average-transaction-completion time is relevant. As you pointed out, the bitcoin stats about data exchange are relevant too.)
Thank you for sharing the URLs. I will think about it a bit more and get back to you."
Bebo emailed:

  • More and more I am convinced that we could identify an interesting and relevant project taking advantage of our PingER experience and involving Blockchain technology.I also think that it may be possible for us to get funding/grant money for such a project which might clearly draw the attention of Blockchain entrepreneurs, conferences, and publications. It might also allow us to attract the collaboration of other institutions and/or universities that might otherwise have not been interested in the basic PingER goals and technology.

Though the transactional latency is important especially for achieving consensus, most of the latency is computational that is much greater than the communications latency. Possibly PingER RTT could be included in a measure of BlockChain health and status.  Bebo sent a reference (https://mastanbtc.github.io/blockchainnotes/consensustypes/) pointing out milliseconds may make a difference to miners to get rewards which is important for BlockChain cash
Saqib sent email of an analysis of the impact of various RTTs, see https://ethereum.stackexchange.com/questions/18201/does-network-latency-significantly-affect-mining-rewards.
A possible problem is finding the IP addresses of the bitcoin miners. Bebo dug up a URL showing how it can be done, it is at https://bitcoin.stackexchange.com/questions/66260/finding-ip-addresses-of-all-bitcoin-miners.

Bebo's impression is that Saqib will lead in putting the ideas in his paper into practice. Saqib will need some students.  Saqib's boss is going to the NY City meeting.

There was a meeting to discuss blockchain possibilities, see 20180709 PingER Meeting on Blockchains. 

  • The hope is that Saqib (or someone at Guangzhou University) will continue to explore this topic.
    • Saqib is pursuing PingER and Blockchain. He is looking at different references shared by Prof. Bebo and the implementation details using Hyperledger Fabric.
  • Also, we are curious concerning the response to the paper that was presented at the conference in New York.

PingER at SLAC (Updated 6/7/2018, no update 8/9/2018)

Umar looking at extending the comparison IPv6 vs IPv4 ping RTTs and TCP vs ICMP/ping RTTs. 

  • See Towards Analysis of ICMP vs TCP Ping Latencies - Umar
    • Looked into Traffic Differentiation - Rate Limiting vs. Traffic Prioritization (QoS)

    • IPv6 results gathered using ping-vs-tcp.pl script. About 56 nodes with IPv6 addresses, 14 of which responded with Npings

    • IPv4 results gathered from SLAC and Virginia Tech
      • SLAC's batch may be downloaded here (approx. 24 MB)
      • Skimmed results; findings are pretty much the same as before
  • Pending
    • Identified relevant events in the network stack that highlight timing (_RECVFROM, _RECVMSG, _IP_RECV, _NETIF_RX etc.). Looking for instrumentation that enables us to measure timestamps. We also need to figure out how to determine whether ICMP & TCP traffic are treated differently? and then how to measure the difference?
      • perf-tools allows us to measure transport events
      • If we could assume that the path for ICMP & TCP through the network is the same, then the only difference between two (controlled) tests would be the time spent in the transport layers. This can be measured using perftools. 
      • However, such measurements must be made in a controlled environment where ICMP and TCP are treated the same. (I say so because some results — e.g., in East Asia and South Asia — clearly show that ICMP performs much worse than TCP.)
    • We would also need to cater for cross traffic and queuing delays. Given how small the differences are, one may argue that the variations in measurements are due to cross traffic. Perhaps we should start with controlled tests and then see if real world measurements reflect similar behavior.
    • We need to setup a test environment. We can either setup a bare-metal box or use a VM. 
      • I will see if I can arrange for a bare-metal box.

PingER IPV6 support

  • Les identified and sanitized a Cross Site Scripting (XSS) exposure in ping-data.pl. He will send out a notice to please update.
  • Les is cleaning up the Beacons (i.e. removing non-responders and replacing, adding Beacons for countries with no Beacons).
HostStatelast seenStatus
pinger.ascr.doe.govDown since 21st July with a bad disk, contact will move MA to a VM7/20/2018 
pingersonar-um.myren.net.myNo response6/26/2018Pings
pinger-ncp.ncp.edu.pkpinger-ncp.ncp.edu.pk downNov 29, 2017 
121.56.146.180 (pinger.kohat.edu.pk) DownNov 22nd, 2017 
cae.seecs.edu.pkDownFeb 27, 2018 
pinger.isra.edu.pkDownMarch 6, 2018 
pingeramity.inIt has been working for since 28ty July. It is unclear how stable it is.April 27, 2018 

Next Meeting

Next meeting:  Thursday, August 9th 9 pm Pacific time; Friday, August 10th, 2018 9:00 am Pakistan time; 12:00 noon Malaysian & Guangzhou time; and 11 am Thailand time.

...

GZHU China - Saqib (moved here 7/2/2018)

...