...
Wednesday Aug 24 9:00pm Pacific Standard Time, Thursday Aug 25, 2016 9:00am Pakistan time, Thursday Aug 25 2016 12:00noon Malaysian time, Thursday Aug 25 2016 02:00am Rio Time.
Coordinates of team members:
See: http://pinger.unimas.my/pinger/contact.php
Attendees
Invitees:
Hassaan Khaliq, Muhammad Anas Abrar+ (SEECS); Saqib?, Aqsa (UAF); Johari?, Adnan Khan (UNIMAS); Badrul, Ridzuan, Ibrahim (UM); (UTM); Adib?, Fatima (UUM); Fizi Jalil (MYREN); Les+, Bebo, Joao +(SLAC).
...
- Responded but Unable to attend:
? Individual emails sent
Actual attendees:
Aib, Saqib, Joao, Les, Bebo
Administration
PingER for Android (see ePingER on Android two methods).
Amity - Java approach (A. Sai Sabitha and Shivnarayan Rajappa)
- They are using the native java tools, they are not running the pinger2.pl <http://pinger2.pl> script on android since the native java tools have the following advantages
- easier for user,
- no need for prior installation of any software, e.g. load perl interpreter which may require missing skills, especially for a non technical user
- doesn't need a rooted phone
- only the apk needs to be installed to run
They have now updated the underlying ping command to ping -n -c 10 -w 30 -i 1 -s 100 www.google.com <http://www.google.com> this now allows them to collect 10 pings with 30 pings being the maximum limit. The attached text file (data.rtf) shows a sample situation wherein 11 packets are sent and 10 are received where packet 3 was lost. This was not possible in the previous implementation.
There is a bug in the sequence number reported, they are looking at.
They have also tested all the other edge cases and use regex to validate the ping data received before writing it to the text file. If they get an unknown host it fails the regex check therefore is not written into the text file.
Next steps:
Extend extend the target list by getting the Beacon list from SLAC. It is at http://www-iepm.slac.stanford.edu/pinger/pinger.xml on a regular basis and updating the <BeaconList> section at their site. This was part of pinger2.pl.
- Also they will need a utility to clean out old recorded data (say older than 3 months), since it will be gathered from SLAC (via the proxy) and eventually they may run out memory on the Android.
UAF Saqib and Sara Masood
- They are using the original pinger2.pl script.
- This guarantees a better, more complete (logs, checks, updating the Beacon list etc.) match to the existing MAs.
- There is a tutorial on the android version.
- They are working on getting a public fixed IP
Discussion
To a large extent it depend on how we plan to use this.
...
A next step is to get the data from the phone MA to the archive at SLAC. The current method ping_data.pl requires a public IP address for the phone which may not exist if its is mobile. Getting the MA to put the data to the archive may raise some security issue for the archiver.
UUM
From Adib:
Adib is blessed with a baby girl (Maryam) on Sunday 18, August. Currently on parental leave, he will try to attend.
Adib enquired about the status of his request to visit SLAC. After the meeting Les followed up with SLAC's international services, they said requests are taking 3-9 months to process.
Adib asked a student (Jaafrau) to explore “Creative visualization of PingER data, including rich interaction;”. He has come up with a good design, yet the development part needs a lot of work. Adib will ask him to upload his interface and share the link. No update 8/24/2016
Adib has another student starting in September. Adib will look at directing him to PingER.
a student from Thailand who is working with Adib at UUM who may be interested in installaing a PIngER MA in Thailand. Adib had a meeting with Mr. Chaisit Choosong <chaisitc@gmail.com>, from Rajamangala University of Technology Srivijaya last week. Adib gave him a brief about PingER project, aim, objectives, and its installation requirements. He is interested to setup a PingER monitoring node at his site. Adib sent another email to introduce Chaisit to Dr. Les and hopefully, Les has followed up with an email on 8/21/2016 with an invitation and information on how to set up the monitor. Adib will follow up.
Adib has written a case study on the Internet performance in ASEAN countries and its implication on the Socio-Economic Indexes.
Following Prof. Bebo suggestion to find one of the IEEE journals that have good exposure in Asia. Adib used IEEE Publication Recommender tool and got a long list of journals within our article scope. However. They have page limits or # words limit. We do not have many choices; here is a short list of most potential ones:IEEE Technology and Society Magazine: “the scope of IEEE Technology and Society Magazine overlaps most IEEE Publications since we are concerned with the social implications of technology across the wide range of IEEE fields of interests.” http://ieeexplore.ieee.org/xpl/aboutJournal.jsp?punumber=44Editor-in-Chief
Katina Michael
School of Information Systems and Technology
University of Wollongong
NSW 2522 AustraliaIEEE Communications Surveys & Tutorials: http://ieeexplore.ieee.org/xpl/aboutJournal.jsp?punumber=9739Editor-in-Chief
Ekram Hossain, Ph.D., P.Eng.
Professor
Department of Electrical and Computer Engineering
University of Manitoba
UNIMAS
...
UAF (Saqib)
There are 4 students: Aqsa Hameed, Saba Muzamil, Tahseen and Sara Masood. T
...
- Sara Masood - See PingER on Android above
Saqib will proceed to install Pinger in UAF Pakistan 6/22/2016.There used to be an MA at UAF Les will send the information to Saqib. it is below (Is there an update?):
Code Block NODENAME: uaf.seecs.edu.pk IPADDRESS: 121.52.152.22 SITENAME: seecs.edu.pk NICKNAME: PK.UAF.SEECS.EDU.N1 FULLNAME: University of Agriculture, Faisalabad LOCATION: Faisalabad COUNTRY: Pakistan CONTINENT: South Asia LATANDLONG: 31.433741 73.070476 PROJECTTYPE: D PINGSERVER: http://uaf.seecs.edu.pk/cgi-bin/traceroute.pl?function=ping TRACESERVER: http://uaf.seecs.edu.pk/cgi-bin/traceroute.pl? DATASERVER: http://uaf.seecs.edu.pk/cgi-bin/ping_data.pl? URL: http://www.uaf.edu.pk CONTACTS: Ahsan Raza Sattar , COMMENTS: Add by Cottrell 11/5/2012. Host not working as a monitor and not pingable. There is another Monitor in faisalabad, so Disabled by Cottrell 6/26/2014.
MYREN
pingersonar-usm.myren.net.my was restored to a working state August 4th. pingersonat-utm.myren.net.my is having problems, email sent to Fitzi.
PingER at SLAC
Joao has provided a cronjob to automatically update the PingER hourly data on. He has also provided for a client to get new data via the web. Needs documenting.
Also has a 4 node cluster with Cloudera manager for Hadoop, Next step are to get impala working for queries.
Working on the following hosts to be able to gather data
Host | State | last seen | Status |
---|---|---|---|
www.univ-ouaga.bf | Unreliable, No data since June 20th. | June 2016 | Does not ping |
www.ihep.ac.cn | Email 8/3/2016, ping_data.pl works but there is no data, fixed 8/7/2016 | July 28th, 2016 | ping_data.pl works but there is no data |
perfsonar-unimas.myren.net.my | No data August 2016, pings., Emailed Fitzi 8/20/2016 | 7/31/2016 | Not pingable |
pinger-sonar.utm.myren.my | Fixed 8/19/2016, emailed Fitzi 8/19/2016 | 8/9/2016 | Fixed |
pinger.unimas.my | No data since 8/6/2016, emailed Johari 8/20/2016. | 8/6/2016 | Not pingable |
NUST
...
- pinger.isra.edu.pk # http://pinger.isra.edu.pk/cgi-bin/ping_data.pl works but the xml config file contains no beacons. See FAQ at http://www-iepm.slac.stanford.edu/pinger/faq.html#nodata on how to fix
The following 8 .pk hosts are still working:
- comsatsswl.seecs.edu.pk
- monitor.seecs.edu.pk
- nukhimain.seecs.edu.pk
- pinger.aiou.edu.pk
- pinger.nwfpuet.edu.pk
- pinger-ncp.ncp.edu.pk
- pinger.vu.edu.pk
* This host was noted at the 3/9/2016 meeting as being down for the last 30 days.
+ Not seen in June 2016
^ Not seen in July 2016
Les proposes 7/30/2016):
- Give up (i.e. remove from Monitoring node table by setting Projecttype to D (for Disabled)) on non-responsive hosts not working for 2 months.
- I would not oppose extending this to all non-responsive, non-working hosts.
- Focus on the close to working hosts
UM
8/20/2016 Given up.
Next Meeting
Next meeting: Wednesday Sep 21 9:00pm Pacific Standard Time, Thursday Sep 22, 2016 9:00am Pakistan time, Thursday Sep 22 2016 12:00noon Malaysian time, Thursday Sep 22 2016 02:00am Rio Standard Time.
...
Old Items
Visualization ideas for PingER moved here 5/20/2016
...
- article on PingER and African Internet performance, see http://www.huffingtonpost.com/david-tereshchuk/a-giant-leap-in-2016-africa_b_8901556.html
Anjum said it would be good for Hassaan to attach this to his proposal. Any action?
Les is working with the SLAC CIO to try and put something together for publicity concerning the life of PingER.
UOA (Saqib) placed here 2/3/2016.
Saqib has a 5 MS students from the Database team
...
Jan 5, 2015 Hassaan reports "I have received revisions on my proposal and these days I am revising my proposal. In the meanwhile, I have also added another student (Anas Abrar) on this project. He is in learning phase and will follow the nodes which are not working. I shall give you an update very soon. "
- Hassaan is very hopeful that if the proposal is accepted then we can easily have a full time RA for the project.
Hassaan has re-submitted the proposal after revisions. He would like to get Anas Abrar more trained on monitoring operation and then will inform us to add him to the mailing list at http://pinger.unimas.my/pinger/contact.php.
Oct 2015. Following the last meeting, Anjum, Hassaan and Les met to discuss the way forward.
- "Adnan currently is unable to find resources for handling the project. Similarly, there is no progress on hiring of a full time RA by NUST HQ.
- However, I (Hassaan) checked from HEC about the proposal that I submitted last year. They have informed me that 2 reviewers have asked for revisions while they are waiting for the third review. I am very hopefull about it. If the proposal is accepted then we can easily have a full time RA for the project. I have plans to talk to Dr. Zaidi about hiring an RA on assuming that our proposal will be accepted by HEC. We can then get his salary deducted later from the HEC project. I shall update you very soon in this regard."
Hassaan is waiting to hear from HEC about the comments on the proposal.
- Moreover, he has asked a student to work on the project for the time being. His name is Mian Anas however he will need few weeks to understand the project.
Thiago completed setting up the PingER data SQL Impala warehouse running on a Nebula/Cloudera cluster using the Hadoop File System (HDFS). Unfortunately it is not currently accessible from outside SLAC. There have been several attempts to provide outside access, but no success yet, we need to engage the subject matter experts. Thiago is now a SLAC associate so he still has an account at SLAC. There was a cyber security alert on the version of java installed with Cloudera. Les has replaced the cloudera version of java which should fix the vulnerability. However the new version has not been tested.
Geolocation
Anjum believes the TULIP Geolocation application can be improved significantly. At least there are few ideas that we can try. For this, either a group of undergraduate students or an active masters student is required. The resultant work can easily be the thesis of masters level. Who is interested?
Saqib at Faisalabad has an MS student interested to work on Geolocation project. He requests an initial paper on the project. Les has responded to Saqib. He also has some other students. Anjum will contact him. Potential projects/asks include: take over management of PingER monitoring in Pakistan (say 5 monitors/student; case study of how Pakistan's network performance/connectivity has improved over thea years especially as function of funding etc; geolocation with variable alpha; indoor geolocation
Johari will contact Anjum to learn more of the requirements. Update Johari/Adnan
See http://www.slac.stanford.edu/comp/net/tulip/. Basically TULIP uses pings to a target from landmarks at known locations and converts the minimum RTTs to estimate the distances. Then uses the distances with mulitlateration to estimate the location of the target
To improve TULIP one needs the right selection of landmarks, i.e. good (working landmarks) at the right locations (not too far from the target), straddling the target, and with a a reasonable estimate of the indirectness (directivity or alpha) of the path from the landmark to the target (so we can reasonably accurately estimate the distance). One also needs a reasonable density of landmarks (e.g. number of targets/100,000sq km)
The landmarks come from PingER and perfSONAR sites. We have a reasonable density in the US, Pakistan and Europe. Currently Anjum is getting better than 20km accuracy for Pakistani targets
As the number of landmarks goes up so does the accuracy, but so does the time to make the measurements (pings).
One needs to find the optimal density
Anjum proposes to speed up the measurements using a cluster for parallelization and also proposes to improve the adaptation of alpha based region. He regards the adaptive geolocation and parallelization as MS projects.
He is also interested in geolocation in small proximity (e.g.indoors), e.g. using cell tower signals. This is a new area of research. It is possible that the port of PingER to an Android could be related to this. This is a PhD project
Anjum reports he can supervise the students on Geolocation. He will need to know when the students are ready. We can start with a joint meeting involving Les and the students. Later on, Anjum can have the meeting with students every week while Les can join if he has time.
NUST/SEECS Pakistani PingER nodes status
Pink Background indicates host was bad last month, strike through says it is fixed, yellow is an new bad host.
...
Is it time to start paring down the list of PingER monitor hosts in Pakistan, starting with those that have been down for a while and despite your efforts they are not cooperating. One might also look at the coverage by region in Pakistan and try and keep good coverage for all regions.
Traceroute at UTM 5/9/2015
The traceroute problem regarding maximum reachable hops ( i.e. 11 hopes ) may be since the Unix/Linux/OSX traceroute uses UDP to send the requests. The first request is sent to a particular port (33434), with a ttl to tell it how many hops to go to. The ttl starts at 1 is incremented as it tries the next hop, also the port is incremented (up to 33465). It looks like the first few UDP ports are enabled and then they are blocked. The Windows traceroute uses ICMP to send the probes so does not see the problem.
Linked Open Data
Cristiane reports (7/1/2015): "I am trying to automatize the triplification of PingER data on Kettle. For now, part of the transformation is made on Kettle and another is made by a Java code. Although this solution works for a data sample, is important to have the entire process on Kettle because it facilitates to understand, modify and control the triplification process."
...
Christiane's report is at: Size Inflation of PingER Data for use in PingER LOD
UM
Moved here 3/4/2015:
Ibrahim has setup distributed hadoop clusters. He has 2TB of disk space. Les has provided information on getting a subset of PingER data by anonymous ftp via ftp://ftp.slac.stanford.edu/users/cottrell. It was put there last September. Information on how the data was put together is at https://confluence.slac.stanford.edu/display/IEPM/Archiving+PingER+data+by+tar+for+retrieval+by+anonymous+ftp. There is information on formatting etc at http://www-iepm.slac.stanford.edu/pinger/tools/retrievedata.html and some on the dataflows at https://confluence.slac.stanford.edu/display/IEPM/PingER+data+flow+at+SLAC. Renan at UFRJ has successfully used this data, he has also characterized the data in terms of bytes/metric per year etc.
...
Anjum reported that UM had experienced a TCP syn DOS attack prior to Mar 12th (when an IDS was put in place). It occurred mainly for several days before between the hours on noon- 2pm and 7-7 in the evening (Malaysia time). He suggested looking to see if PingER could spit the effect. Ibrahim, Les and Anjum will look at. Les analyzed the data and sent it to Anjum
NUST
The following is from Samad 2/24/2015.
- buitms.seecs.edu.pk #We have to disable gathering data from this host because the person still don't want to continue with us as i have tried once again to convince him but the answer is same. Les has disabled from SLAC.
- nukhimain.seecs.edu.pk # We were unable to gather data since 20th November, 2014 and now the Node is working fine and collecting data as well.
- pinger.uettaxila.edu.pk #The node is working fine from last two weeks.
- sau.seecs.edu.pk. #This Node is working fine now.
- pingerjms.pern.edu.pk #This node is working now.
- pinger.uet.edu.pk # this was also not working from so many days. and now its working fine and collecting data as well.
- pinger.isra.edu.pk # This node is also working fine now.
- pingerlhr-pu.pern.edu.pk # This is also working fine now.
- pinger.kohat.edu.pk # Collecting data now.
The IP of "pingerqta.pern.edu.pk" has been changed, Les has updated the databas at SLAC with the following
Old IP: 121.52.157.157
New IP: 121.52.157.148
Follow up from workshop
- Hossein Javedani of UTM is interested in anomalous event detection with PingER data. Information on this is available at https://confluence.slac.stanford.edu/display/IEPM/Event+Detection. We have sent him a couple of papers and how to access the PingER data. Hossein and Badrul have been put in contact. Is there an update Badrul?
...
Anjum suggested Saqib, Badrul and Johari put together a paper on user experiences with using the Internet in Malaysia as seen from Malaysian universities. In particular round trip time, losses, jitter, reliability, routing/peering, in particular anomalies, and the impact on VoIP, throughput etc. It would be good to engage someone from MYREN.
Ibrahim
Ibrahim Abaker is planning to work on a topic initially entitled " leveraging pingER big data with a modified pingtable for event-correlation and clustering". Ibrahim has a proposal, see https://confluence.slac.stanford.edu/download/attachments/17162/leveraging+pingER+big+data+with+a+modified+pingtable+for+event-correlation+and+clustering.docx. Ibrahim reports 7/15/2014 "I have spent the last few months trying to understand the concept of big data storage and its retrieval as well as the traditional approach of storing RDF data. I have integrated a single hadoop cluster in our cloud. but for this project we need multiple clusters, which I have already discussed with Dr. Badrul and he will provide me with big storage for the experiment." No Update 8/20/2014.
"I have come up with initial proposed solution model. This model consists of several parts. The upper parts of the Figure below shows the data source, in which PingER data will be convert into RDF format. Then the data pre-processor will take care of converting RDF/XML into N-triples serialization formats using N-triples convertor module. This N-triple file of an RDF graph will be as an input and stores the triples in storage as a key value pair using MapReduce jobs"
Potential projects
1) pingER monitoring host on android .
...