Time & date
Wednesday April 23, 2014 9:00pm Pacific Standard Time, Thursday April 24th 2014 9:00am Pakistan time, Thursday April 24th, 2014 12:00 noon Malaysian time, Thursday April 24th, 2014 1:00am Rio Standard Time.
Attendees
Invitees:
Anjum, Hassaan Khaliq, Kashif+, Raja+, Johari+, Nara, Abdullah, Badrul, Ridzuan+, Ibrahim, Hanan, Saqib+, Adib, Les+, Renan, Bebo
+ Confirmed attendance
- Responded but Unable to attend:
Actual attendees:
Kashif (but lost power), Raja, Johari, Ridzuan, Saqib, Les
Administration
Adib is waiting to hear from colleague (Dr. Nordin: chair of ISTT2014), there was no update as of this meeting. The conference is 2 days so the workshop may be squeezed to one day. The conference will be held in Langkawi between 24-26 Nov.
Hopefully, this will help us to attract good number of participants and get sponsor/support to cover our expenditure. Other option is to conduct the workshop in UUM, we are ready to provide Local Arrangement (lab, local transportation, and food). But there is no funding provided by the school to support the transportation/accommodation of the speakers.
Les will be in Burkina Faso Nov 13-21. He flies back to London on Nov 22nd. He can get to Malaysia mid morning November 24th. Thus the workshop should be at the end of the ISTT2014 meeting if Les is to attend.
Renan
Renan 's paper on "Linked Open Data Publication Strategies: Application in Networking Performance Measurement Data" submitted to the Academy of Science 2014 Bigdata / SocialCom / Cybersecurity conference at Stanford May 27-31 has been accepted for a poster session. He is looking into travel funding, else Bebo and Les will try and cover. Renan has to respond in the paper to some questions raised by reviewer(s).
UM
Badrul is still awaiting hearing from his student (Abdulrahim Haroun Ali) on the paper on anomalies in PingER measurements and will update later once the paper ready.
Ridzuan has put together a rough proposal to use Hadoop to store and make available PingER data. There is a rough draft at: https://confluence.slac.stanford.edu/download/attachments/17162/leveraging%20pingER%20big%20data%20with%20a%20modified%20pingtable%20for%20event-correlation%20and%20clustering.docx. Ridzuan requested a copy of the PingER data. Les & Raja put together a repository of the minimum_rtt by hour data since Jan 1998 and made it available by FTP.
There was a discussion about using hadoop and other hosts joining. Also the MYREN cloud has public access to Malaysian academic institutions may be of interest. Johari will mail some details to Ridzuan.
Ibrahim Abaker is planning to work on a topic initially entitled " leveraging pingER big data with a modified pingtable for event-correlation and clustering". Ibrahim has a proposal, see https://confluence.slac.stanford.edu/download/attachments/17162/leveraging%20pingER%20big%20data%20with%20a%20modified%20pingtable%20for%20event-correlation%20and%20clustering.docx. He is currently spending time reading about RDF storage and its retrieval, also he is trying to setup Hadoop clusters for the experiment. He is in email discussion with Renan on which part to work on first. He plans to collaborate with Renan to make the pinger store and process more efficient. Les has sent him documentation of PinGER which is very helpful. Ridzuans' work is more to do with hosting the data and stream data analysis. Ibrahim is looking more at applying MapReduce ( programming model for processing large data sets with a parallel, distributed algorithm on a cluster), reducing the storage needs and providing querying of the data.
UNIMAS
Dr. Adnan Shahid Khan who recently joined UNIMAS, will be joining our PingER group and will help with the current PingER initiative in UNIMAS. His contact details have been updated at the PingER Malaysia website (http://pinger.unimas.my/pinger/contact.php). Badrul has been requested to add Adnan's email address to the pinger mailing list.
The Raspberry Pi is at the data centre and has a public IP address. The private IP address works but not the public. Johari reports no progress 4/8/2014, 4/23/2014.
The tool to enable synchronizing Malaysian monitors is completed. It has been tested by Saqib. Saqib requested to add sorting the HostList by country, Johari will add this.
The traceroute server at http://pinger2.unimas.my/cgi-bin/traceroute.pl has the same problem as before. They know (sort of) the problem but haven't got the chance to rectify it (mapping NAT address, needs to be added). There is no progress 12/4/2013, 1/8/2013, 1.22.2014, 2/5/2014, 3/26/2014, 4/8/2014, 4/23/2014. Now that the historical traceroutes are working for UM (see below) there is an extra incentive to get the reverse traceroute working at UTM and UNIMAS
Johari has a research student who finalized a proposal in order to officially apply for his masters. He will start in February. He is currently working on threshold/anomaly detection, and will extend to correlating performance over multiple routes. He will share the proposal with Les and others April.
UTM
Saqib is seeing some unexplainable results from the traceroute servers at UTM. Basically traceroute from UTM works fine with -I option. Otherwise it stop at hops 10 or 11 unless he uses the -f 6 option. He documented by email and Les added it to the wiki at Traceroute at UMT. The traceroute problem is under discussion with MYREN and CICT UTM.
Saqib had a detailed discussion regarding the Packet loss at UM with Myren. They suggest that packet loss January 11-13 of >4% loss may be due to congestion in the UM edge router (perhaps due to high traffic or traffic filtering/shaping). They suggest to pursue this with UM IT Division to know what is hop 11. Saqib has emailed Badrul.
The Case study is almost done. Saqib has included the packet loss analysis at Um in the case study. Waiting for the reply of MYREN and CICT to include in it. Need one more week.
Saqib received feedback on the submitted FRGS 2014 proposal ("A Fundamental Active Internet Performance Monitoring Framework for Malaysian Research & Education Network") for 36 months. He will update it and re-submit. He proposes to submit another in ERGS or E-Science grants. He is in discussions with his PhD. superviser to get the requirements and has started on the proposal.
UUM
Regarding the monitoring host in UUM, Adib has assigned one student to prepare the configuration/installation plan including how to secure our host from attack. He has a public IP address. He needs to the DNS registration by Sunday or Monday. He is in the last stage of working with the Computer Center. Adib requested Johari to share the UNIMAS setting so it is easier for the student to follow.
NUST
Kashif has restored Maggie1 to a working state. It was down due to hard disk problem since 11th April. New installation on a new disk has been made and its working now as a monitoring node.
Kashif has copied new beacon list (modified pinger.xml for pakistani nodes) at the HEC-PoP node at Khi (pingerkhi.pern.edu.pk). By looking at the data it seems to be allowing pings outside Pakistan. This implies, its working fine and there is no blockage (May be for some outside Pakistan nodes but not for all). So the new synchronized beacon list has no problem so far.
All references for the Bahawalpur site goes to Director IT, but he is still silent and we are waiting for his reply.
Kashif reports the installation of PingER at Sahiwal is complete. Raja will add to PingER and TULIP (site is not responding - might be a power outage 11:42AM 4/23/14 Pacific Time).
We are unable to gather data from airuniversity, cae, duhs, kohat, lcwu, pinger.nca.edu.pk, pingernwfpuet.edu.pk, pinger.pern.edu.pk, pingerfsbd, pingerisl-fjwu.pern.edu.pk, pingerisl-qau, pingerkhi-uok.pern.edu.pk, pingermtn.pern.edu.pk, sau.seecs.edu.pk , uaf, www.upesh.edu.pk.
Monitors Status
Airuniversity New installation will be install in a week
Buitms Problem in hardware, they are trying to fix it. System Replacement Required
Cae Problem with IP, they are trying to resolve it
Duhs They are purchasing new servers in two week time
Pingerisl-qau Contact person is still on leave. He is now busy in other pending works.
sau.seecs.edu.pk for all of January. Contact person is not attending
Uaf Problem in system, visit approval for Faisalabad University in progress. Approved for 12th Feb
Anjum pointed out that there are working PingER monitors that SLAC appears unable to gather data from.
Raja
Raja, Umar and Les are working on a paper on TULIP and its application to Visual Tracerouting to be submitted at end of April.
PingER at SLAC
Saqib discovered a problem with smokeping (accessed from the left hand column of pingtable.pl). Raja fixed it. See http://www-wanmon.slac.stanford.edu/cgi-wrap/smokeping_graph.cgi?destlink=pinger.slac.stanford.edu_pinger.unimas.my_100_10&starttime=1389139925&endtime=1397779925
Les requested an update from Yahoo about TULIP's gelocation. They answered "We are very much interested in getting IP triangulation at internet scale, we will have internal sync-up on how we can leverage this initiative if there is rate limit and get back. Regarding opening up yahoo sites for deploying ping server requires some more time to discuss this with relevant stake holders with in yahoo." No word
Les sent email to Google as follows: "I would like to bring to your attention that we have developed a geolocation tool using delay based (using RTTs from known ping server landmarks) distance estimates to triangulate the location of an IP host target. The tools is accessible at: http://www-wanmon.slac.stanford.edu/cgi-wrap/reflex.cgi. We have identified that the accuracy of the geolocation is directly related to the landmark density (e.g. # of landmarks/ million sq km). The higher the density the smaller the error. We currently have over 1000 registered landmarks, of which at any given time ~300 are working. The tool not onlyfinds the location of the target, it also gives an estimated error. To the best of our knowledge it is the only freely available delay based measurement geolocation tool publicly available today. A drawback (compared to database methods such as those based on GeoMind) is the time taken to make the measurements. We have worked in this from manydirections including parallelization of the ping requests, caching, tiering to get the rough location (i.e. region of the world) then zooming in using all landmarks in the region. We are putting together a publication on this."
Les contacted the Hong Kong, Rutherford Lab (.uk), Daresbury Lab (.uk), CEFET (.br, UNESP(.br) and ACMESecurity (.br) PingER monitors to try and get them fixed. The Hong Kong contact has retired. The others have not responded. Les will probably disable the nodes.
Old Items
Linked Open Data
Renan finished the new pingerlod web site. The new thing is that it should be much easier now to modify the info texts. What Renan did was to put the texts into a separate file. The new version has been loaded on the server and some text added to describe how to use the map. However there is a bug that prevents it from executing the map. Renan reports that the bugs should be easy to fix. He has talked to his professor who suggested trying RDF Owlink, it should have faster responses to queries. Renan will research this. It will probably mean reloading the PingER data so is a lot of work, hopefully this will improve performance. Before the rebuild he will make the fixes and provide a new WAR for us to load on pingerlod.slac.stanford.edu. He is also working on documentation (he has finished the ontology and has a nice interactive tool for visualizing it, since the ontology is the core of the data model of our semantic solution, this will be very helpful for anyone who uses our system, both a developer of the system and a possible user) and his thesis. Bebo pointed out that to get publicity and for people to know about the data, we will need to add pingerlod to lod.org.
Things he will soon do regarding documentation:
- A task/process flow writing all java classes involved on all those batch jobs;
- A Javadoc <http://www.oracle.com/technetwork/java/javase/documentation/index-jsp-135444.html> which will explain all classes and how they are used.
For the Linked Open Data / RDF which is in pre-alpha days, you can go to http://pingerlod.slac.stanford.edu. As can be seen this page is not ready for prime time. However the demos work as long as one carefully elects what to look at:
- Click on Visualizations, there are two choices:
- Multiple Network Metrics: Click on the image: gives a form, choose from Node pinger.slac.stanford.edu pinging to www.ihep.ac.cn, time parameters yearly, 2006 2012, metrics throughput, Average RTT Packet loss and display format Plot graph, then click on submit. In a few seconds time series graph should come up. Mouse over to see details of values at each x value (year).
- A mashup of network metrics x university metrics Click on image: gives another form, pinging from pinger.slac.stanford.edu, School metric number of students, time metric years 2006 2012, display format plot graph, click on submit. Longer wait, after about 35 seconds a google map should show up. Click on "Click for help." Area of dots = number of students, darkness of dots = throughput (lighter is better), inscribing circle color gives university type (public, private etc.) Click on circle for information on university etc.
- Renan will be working on providing documentation on the programs, in particular the install guide for the repository and web site etc. This will assist the person who takes this over.
Renan is using OWLIM as RDF Repository. He is using an evaluation version right now. Renan looked into the price for OWLIM (that excellent RDF Database Management System he told us about). It would cost 1200EUR minimum (~ 1620 USD, according to Google's rate for today) for a one time eternal license. It seems too expensive. No wonder it is so good. Anyhow, he heard about a different free alternative. Just not sure how good it would be for our PingER data. He will try it out and evaluate. He will also get a new evaluation of the free OWLIM lite.
He has also made some modifications on the ontology of the project (under supervision of his professor in Rio) hence he will have to modify the code to load the data accordingly.
Renan has provided a 4 page Appendix on PingERLOD to the ICFA report. This is also available at PingER LOD Overview
Raspberry Pi
A quick comparison of the performance of the two hosts (raspberry pi and regular UNIMAS host) without statistical quantification is available at https://confluence.slac.stanford.edu/display/IEPM/Comparison+of+PinGER+RTTs+from+UNIMAS+monitors+N4+and+RASPBERRY. A page has been created to compare the hardware spec between the pinger.unimas.my node (Intel architecture) and the pinger2.unimas.my node (Raspberry Pi ARM architecture), available from the unimas pinger website at http://pinger.unimas.my/pinger/hardware.php. There is a link to hardware.php in the Comparison+of+PinGER+RTTs+from+UNIMAS+monitors+N4+and+RASPBERRY web page.
NUST
At the Connect Asia Pacific Summit in Bangkok in January and seeing the project "Mapping the pan Asia Pacific information Superhighway and closing gaps in infrastructure connectivity" Shahryar found that very much related to the work in the PingER project. So Shahryar sent email to a UN agency for a possible collaboration with them on PingER project. He has heard nothing so he will write a detailed proposal and then should contact them again. No update 2/5/2014, 3/5/2014.
Tulip
Follow up from workshop
- Hossein Javedani of UTM is interested in anomalous event detection with PingER data. Information on this is available at https://confluence.slac.stanford.edu/display/IEPM/Event+Detection. We have sent him a couple of papers and how to access the PingER data. Hossein and Badrul have been put in contact. Is there an update Badrul?
The Next step in funding is to go for bigger research funding, such as LRGS or eScience. Such proposals must lead to publications in high quality journals. They will need an infrastructure such as the one we are building. We can use the upcoming workshop (1 specific session) to brainstorm and come up with such proposal. We need to do some groundwork before that as well. Johari will take the lead in putting together 1/2 page descriptions of the potential research projects.
- Need to identify a few key areas of research related to PingER Malaysia Initiative and this can be shared/publicized through the website. These might include using the infrastructure and data for: anomaly detection; correlation of performance across multiple routes; and for GeoLocation. Future projects as Les listed in Confluence herehttps://confluence.slac.stanford.edu/display/IEPM/Future+Projects can also be a good start and also Bebo's suggestion.
- Need to synchronize and share research proposals so as not to duplicate research works. how to share? Maybe not through the website, or maybe can create a member only section of the website to share sensitive data such as research proposal?
Anjum suggested Saqib, Badrul and Johari put together a paper on user experiences with using the Internet in Malaysia as seen from Malaysian universities. In particular round trip time, losses, jitter, reliability, routing/peering, in particular anomalies, and the impact on VoIP, throughput etc. It would be good to engage someone from MYREN.
Potential projects
Future meeting - Les
Next meeting Wednesday May 7th 2014 9:00pm Pacific Standard Time, Thursday May 8th 2014 9:00am Pakistan time, Thursday May 8th, 2014 noon Malaysian time, Thursday May 8th, 2014 01:00am Rio Standard Time.