...
Wajahat Hussain+ (SEECS), Saqib+ (GZHU); Johari? (UNIMAS); Adib? (UUM); Dr. Charnsak Srisawatsakul? (Ubru), Les+, Bebo+, Umar?
+ Confirmed attendance
- Responded but Unable to attend:
...
NUST: (Updated 7/5/2018, no update 9/8/2018)
No intern has joined Wajahat 's Lab. So there is not much progress. The students are away on summer vacation so no progress.Wajahat proposes to get a list of the new Universities in Pakistan and contact them encouraging them to participate in PingER and set up MA. They have made a list of new university sites, communications networks, Labs in different regions of Pakistan (especially the remote regions) and will make contact. Any updates?
- The list of new universities is ready. Just need resources to engage them.
- Unsure how this is affected by lack of interns.
Discussion item (7/5/2018)
Wajahat is hoping to get few students to work on Master thesis related to pinger data. If possible kindly share ideas related to data analytics which he can share with students.
Wajahat also asked last time:
- “Has any work gone into to predicting the cause of failures(blackout, flood, coup (Turkey)) using pinger data?”
- Les> there have been several case studies looking at the impacts of failures, however nothing on predicting failures. I am not sure how one might use PinGER to predict the cause of a failure. For some cases (e.g. earthquakes, tsunamis, flood, coup), it is unclear to me how Network performance monitoring could add to other means of predicting the cause.
- Wajahat's thinking is along the lines of "Regarding other sources, I was thinking, internet is the pulse of digital world. Other sources require additional setup that might not be a possibility in developing countries. Internet being a necessity and having other uses is still prevalent in these developing countries. Can it be used as a real time sensor."
- Again I am forwarding the question in case others in the team may have some suggestions? Below are some responses:
- Umar mentioned the use of darkfibre testing (e.g strain) and monitoring for earthquakes. There was a recent paper.
- Umar was thinking that using the PingER data to detect and log anomalies and create alerts might be interesting. This would include triaging the logs into 3 tiers of alerts: It requires hysteresis so one does not get continuous alerts forsay something that has gone down.
- Low = Information;
- Medium = needs action but non-critical, create a ticket to be looked at later;
- Severe: e.g. wake someone up.
- This makes a lot of sense for say routers and switches which are usually very critical.
- The PingER measurement script (pinger2.pl) has a lot of logs that are prioritized by severity (e.g. target not responding for various reasons for the last n runs of pinger2.pl.)
- Tiering of events into alert levels would be the focus of the research
- PingER might be an interesting field to try out the marshaling of events and categorizing into alert levels/tiering.
- Possibly mine the data with say Splunk. Might be an interesting project for an MS orPhDstudent.
- Umar looked at this: Splunk would be useful for rudimentary for simple scenarios, but not for detailed work.
- For PingER it might include correlation with feeds from elsewhere (e.g. earthquakes, tsunamis, social unrest)
- Prediction/tracking:
- For unexpected events such as earthquakes/tsunamis, it is doubtful PingER data would be useful for predictions
- For situations that are developing such as hurricanes, civil unrest: knowing roughly where it is developing might make it possible to temporarily increase the density of targets in the region.
- To do this would mean having a lot more targets available worldwide ready to be monitored.
- One would need to be careful to ensure that the extra monitoring does not interfere with Internet access in the area.
- This tracking might be very valuable for less developed areas where there may be few other (non-PingER) measurements available. However, it can be hard to find the coordinates of ping responding targets is such areas. This would need to be one of the tools developed.
- this is affected by lack of interns.
- Any progress?
Wajahat says (7/10/2018)Wajahat says: "I was discussing the blockchain related email that you sent earlier with Dr. Taha (security researcher at SEECS NUST). He was excited about it. He wanted to try a few things and had a few queries which I was not able to answer. He might join us in our future meetings." Any update?
SLAC was unable to gather data from:
- 121.52.146.180 (kohat.edu.pk) down since Nov 22/2017. Wajahat recommends continuing at least until the new student is up to speed (3/8/2018). No data available 3/24/2018.
- cae.seecs.edu.pk last time we were able to gather any data was February 27th.
- pinger-ncp.ncp.edu.pk pings but can't gather data 8/11/2017 and 9/16/2017. Contacted. Pings but can't gather data 10/24/2017. They are in the process of restoring 1/17/2018. Still down February 28, 2018, await new student. (3/8/2018). No data 3/24/2018. Still down 8/6/2018. It has been Disabled so data is no longer being gathered. Should it be re-enabled?
pinger.isra.edu.pkunable to gather data since3/6/2018,also does not ping.It is working as of September 2nd, 2018.- Wajahat says they will get these nodes up. These have been good nodes. They just need the weekly push. NUST will push them soon. No update 8/9/2018.
UAF/GHZU (Updated 78/9/2018)
Saqib's future at GZHU will be much clearer after 3 months. His current contract expires February 2019.
Anomaly detection
...
Blockchain
- Looking into moving PingER to a "blockchain" database good for decentralizing distribution of data. Monitoring sites would then be able to write to a distributed ledger. This would change the architecture to a more peer to peer architecture. It helps with continuity of PingER since reduces dependence on a single site (SLAC). See BlockChain in Future PingER Projects. Bebo sent several references to Saqib who has looked at them. We could start with real-time data without including the whole archive, i.e. in parallel to the continued centrally managed archive. It would be a private Blockchain and hence not be as compute intensive as a public blockchain.
- There was a meeting to discuss blockchain possibilities, see 20180709 PingER Meeting on Blockchains.
- Bebo's impression is that Saqib will lead in putting the ides in his paper into practice. Saqib will need some students. Saqib is OK with this. He has 2 masters students but they are working in different areas. Maybe NUST can assist with this.Saqib's partner gave a talk/paper on work so far at the New York meeting on July 31st. The talk went fine but there were not many comment/questions.
- Saqib is pursuing PingER and Blockchain. He is looking at different references shared by Prof. Bebo and the implementation details using Hyperledger Fabric. Saqib is looking at making a test implementation. The blocksize will be 2MB-10MB. It does not appear to be computationally expensive. He will start testing with Internet of Things measurements such as humidity and temperature
- There was a discussion on the use of DataBases and whether they could be avoided by caches. Hyperlogic keys are not in an SQL DB, basically, it appears like a cache. there was a question whether a 10MB block would be adequate for PingER. For example, PingER from SLAC has about 700+ targets, the measurements are each 30 mins (48/day) and for pings of 100B and 1000 Bytes i.e. 2*48*700 measurements and each measurement is ~ 140Bytes, so a day's sets of measurements from just SLAC is ~ 10MBytes. We could choose to ignore the 1000Byte pings which would reduce it to ~ 5MBytes/day. The latency of retrieving a block is proportional to the block size. Things will be clearer after the test set up is in use.
IPv6 measurements
- There are now several months of IPv6 PingER measurements from GZHU/BJ, UBRU and SLAC. It is time to think about in-depth analysis of the data.
- Umar has not made much progress on the comparisons of IPv6 vs IPv4 or TCP vs ICMP.
- The PingER measurements would provide longer time spans where one might look for changes with time such as diurnal changes, impact of holidays, anomalies etc.
PingER at SLAC (Updated 6/7/2018, no update 8/9/2018)
...
| Host | State | last seen | Status |
|---|---|---|---|
| pinger.ascr.doe.gov | Down since 21st July with a bad disk, contact will move MA to a VM | 7/20/2018 | Fixed August 9th, 2018 |
| pingersonar-um.myren.net.my | No response | 6/26/2018 | Pings |
| 121.56.146.180 (pinger.kohat.edu.pk) | Down | Nov 22nd, 2017 | |
| cae.seecs.edu.pk | Down | Feb 27, 2018 | |
| pinger.isra.edu.pk | Down | March 6, 2018 | |
| pingeramity.in | It It has been working since 28th July. It is unclear how stable it is. It is down again 9/5/2018) | April 27, 2018 |
Next Meeting
Next meeting: Thursday, September 6th 9 pm Pacific time; Friday, September 7th, 2018 9:00 am Pakistan time; 12:00 noon Malaysian & Guangzhou time; and 11 am Thailand time.
...
Old information
Discussion item (7/5/2018)
Saqib sent an email to the team:
...