Time & date
Wednesday January 8th 2014 8:00pm Pacific Standard Time, Thursday January 9th 2014 9:00am Pakistan time, Thursday January 9th, 2014 noon Malaysian time, Thursday 9th January, 2014 02:00am Rio Standard Time.
Attendees
Invitees:
Anjum, Hassaan Khaliq, Kashif+, Raja+, Johari+, Nara, Abdullah, Badrul, Ridzuan+, Hanan, Saqib+, Adib, Les+, Renan-, Bebo
+ Confirmed attendance
- Responded but Unable to attend:
Actual attendees:
Hassaan, Kashif, Raja, Johari, Badrul, Saqib, Les.
Saqib had a poor connection and had to drop out.
Administration
- Do we need to add anyone else to the weekly meetings, e.g. Hossein Javedani of UTM. Saqib has tried to contact Hossein 3 times. We will drop this item.
- Johari proposes that UTM host a PingER workshop in 2014, maybe associated with another conference. Anjum will be in Malaysia then. Saqib will meet Hanan Jan 9th and discuss this. Johari will email Hanan also.
- Anjum is going to spend a year in Malaysia at UM. He awaits his visa. Dr Hassaan Khaliq will be taking over as the PingER Leader there. Badrul agreed to add Hassaan to the pinger-my list. Hassaan's coordinates are in http://pinger.unimas.my/pinger/contact.php.
Renan
Renan's thesis has around 130 pages. However, almost half of it is about generic stuff about Semantic Web. The other half is specific about PingER LOD project. The Semantic Web generic stuff part should be translated because it is good material for those who want to study and understand more about it. The PingER LOD part should be translated for further documentation about the project we developed at SLAC. Things Renan needs to do:
- Start translating his thesis, specifically the PingER LOD part (around 50 pages), which is more interesting to us.
- Test (free) alternatives for the RDF Repository, he has already started this with Virtuoso. He just needs more time to test it more and have more solid results.
-
Basically he would need at least one week to dedicate myself to work only on PingER LOD programming, bug fixes, tests, documentation, and stuff. However, this is currently very hard because of the job he has at his university. IHe says 'I will try my best to find some time, though. Promise!'
Any help from a developer who would be able to engage him/herself to invest some time on the project would be very much appreciated.
-
- Paper about Big Data and Social Computing for the conference that will be held at Stanford: He is pretty sure he can argue that we have made nice contributions for social computing using (big) data handling technologies. he will discuss with his professor at Rio so she can help him with advice. The deadline for the proposal (3 pages, easy) is Jan 15.
- Provide a short description of PingERLOD for the annual PingER report.
Renan is now working as a systems developer at his university at this moment so he is busy. He has plans to keep in touch with us and producing more work. Currently he is unsure he will have an outstanding productivity.
UM
Ridzuan reports his current progress is that he has considered using Apache Mahout (open source machine learning for big data) for the clustering on the Pinger data. Ridzuan, how did your tests with pingtable.pl go? Ridzuan is awaiting Renan decision on final implementation of RDF storage to be adopted.
- Renan reports that Ridzuan can definitely use PingER LOD data because it is constantly being updated, due to the chronological jobs. The main problem he will face is the slowness caused by the usage of Sesame (RDF Repository). The slowness is clearly shown on more complex queries, especially the ones that retrieve measurement data using many parameters (ex: a query that shows throughput, packet loss, and avg RTT from SLAC to all nodes in Malaysia, monthly, since August).
- Ridzuan may find it much faster and more support available if he uses pingtable.pl etc.
- Ridzuan agrees about the slowness. He will use PingER's text files via pingtable.pl etc. Les mentioned there are several for applications to access PingER data via the web.
Badrul plans for his undergraduate student (Abdulrahim Haroun Ali) to have a draft paper on anomalies in PingER measurements to share for review by the end of semester (January).
UNIMAS
Raspberry Pi was repaired with duct tape returned to computer center. However the heat of the device disolved the tape glue. Johari will get a replacment under warranty. The conclusion is that the Raspberry Pi is OK for a contrlled environment but not robust enough for rural areas such as the Kelabit Highlands.
The UNIMAS monitor was down over the new year. The problem was that the pinger2.pl file was moved. It is now working, however it looks like an older version, it does not have the individual ping response just the min/avg/max. This will need investigating.
The tool to enable synchronizing Malaysian monitors is completed. It provides the ability to add, edit and delete nodes. However there are still some errors (wild cards) in the pinger.xml file produced. Johari is working on it.
The traceroute server at http://pinger2.unimas.my/cgi-bin/traceroute.pl has the same problem as before. They know (sort of) the problem but haven't got the chance to rectify it (mapping NAT address, needs to be added). There is no progress 12/4/2013, 1/8/2013.
Johari has a research student who finalized a proposal in order to officially apply for his masters. He will start in February. He is currently working on threshold/anomaly detection, and will extend to correlating performance over multiple routes.
UTM
Saqib has finished his thesis. He is applying for a Postgraduate Research assistant-ship at UTM where he plans to work on PingER. He is still waiting to hear the result.
Saqib is working with the IT department in UTM to solve the problems of delays in traceroute. They have a new ISP. Now the pinger server is working with single static IP (161.139.68.188) both for intranet and internet. Previous internet IP 161.139.146.158 is removed.
- No load balancing is involved as UTM is working with single ISP.
- Still the problem of the delay in traceroute is not solved. It appears to be due to the DNS lookup delay.
Saqib emailed a list of Potential PingER remote nodes in S. E Asia. They have been added to the all 3 Malaysian PingER monitors' HostLists. He has started a case study of Malaysian hosts seen from Malaysian Monitors.
UUM
Dec 4, 2013: The director of computer centre has agreed to support this collaboration and already appointed one staff to be with us during the configuration. Next step, Adib needs to discuss with Prof. Suhaidi to dedicate one PC OR get support from Badrul grant to buy one PC for this purpose or have a bake sale and buy a Raspberry Pi. Adib will do his best to setup PingER monitoring host at UUM before the end of this year. Adib was not on the meeting. He provided an update by email: "Still waiting to get PC. But they promise to provide one very soon. it is totally beyond my control." Regarding the meeting, Adib will attend the upcoming meetings once UUM pinger host is up.
NUST
At the Connect Asia Pacific Summit in Bangkok 2 weeks ago and seeing the project "Mapping the pan Asia Pacific information Superhighway and closing gaps in infrastructure connectivity" Shahryar found that very much related to the work in the PingER project. So Shahryar sent email to a UN agency for a possible collaboration with them on PingER project.
Anjum believes he can add two more PingER hosts, one at Bahawalpur the other at Sahiwal. These are in central Pakistan towards the Eastern border and so should help with providing TULIP landmarks. However with his impending move he has not had time to follow up. Hassaan will follow up with Anjum.
We were unable to gather data from about 13 monitors in Pakistan. Three have been fixed. They will fix the remainder. The contact for QAU is on leave. New DVDs have been sent to DUHs
TULIP - Raja
We are looking to make the code open source and available via github. We will be discussing possible extensions and potential commercialization.
Raja has found a further ~48 perfSONAR landmarks mainly in Europe and N. America. We now have 228 active unique landmarks. He now checks for new landmarks on a monthly basis. We have also created Taiwan as another region for which we can get good location information. We now work for N. America, Europe, Pakistan and Taiwan.
Raja has made lots of measurements and created cumulative probability distribution functions of how the new TULIP with adapting alpha (in distance = Alpha*100(km/ms)*min_RTT(ms)) according to region and min_RTT compares to databases and Constraint Based Geolocation and Single Ping etc. He has also added information on the impact of landmark density on accuracy.
Raja has put together a paper on TULIP. It has been submitted.
As Raja's work on TULIP begins to go into testing he will be engaging more with Renan and Linked Open Data. Raja has sent email to Renan and has a list of things from Renan that Raja should look at. This is delayed by finishing off the TULIP paper and the end of year PingER annual report for ICFA.
PingER at SLAC
Les tried again with UFRJ (12/5/2013), making sure Renan is on the email so Renan can go beat on the door of the UFRJ contact. There was no response. Renan is going to go to the contacts office tomorrow to try and get things moving.
Zakaria of Zayed university in Dubai copied Prof. Wathiq Mansoor of the American University in Dubai. Wathiq responded with interest, Les sent him information (basically http://www-iepm.slac.stanford.edu/pinger/letters. Given the pre-production nature of PingERLOD (it is slowwwww), we will defer engaging them until it is more production like.
We are working on the annual PingER report for the International Committee on Future Accelerators (ICFA). It is due mid January.
We made case studies of: the Democratic Republic of the Congo and Cuba upgrading from GEOS to terrestrial fibre optic links, and a comparison of PingER derived throughputs vs those of Speedtest.net (see https://confluence.slac.stanford.edu/display/IEPM/Comparison+of+Speedtest.net+download+speeds+versus+PingER+Derived+throughputs).
Worked with contacts to fix up monitors at TRIUMF in Vancouver, Canada, DoE ASCR in Washington DC, AMPATH at FIU Florida.
Old Items
TULIP
The new beta test TULIP site is up and running and is at http://tulip.slac.stanford.edu. To first order (due to the number of landmarks available) it only works in N. America, Europe and Pakistan. Even then it is only accurate to a hundred or so km. It also will not work for targets that do not respond to pings or are connected via geo-stationary satellites. Its main use at the moment maybe to find roughly the location, i.e. region/country/state, a target is in. This is particularly useful for proxies and for routers (the latter are typically mis-found by GeoIPtools to be in the corporate HQ of the owner (e.g. Berkeley for ESnet routers). It would be really valuable if router owners provided DNS LOC records filled out.
There was interest in TULIP from Mridul Jain <mridul@yahoo-inc.com>and the Senior Architect in Yahoo! R&D Software Dev in Bangalore, India. He pointed to an interesting paper on Geolocation using CBG and then finding hosts in the area and getting their area codes and using a virtual landmarks making traceroutes. The paper is at https://www.usenix.org/legacy/event/nsdi11/tech/full_papers/Wang_Yong.pdf. He wants something to use in production, however I do not think we are ready.
Linked Open Data
Renan finished the new pingerlod web site. The new thing is that it should be much easier now to modify the info texts. What Renan did was to put the texts into a separate file. The new version has been loaded on the server and some text added to describe how to use the map. However there is a bug that prevents it from executing the map. Renan reports that the bugs should be easy to fix. He has talked to his professor who suggested trying RDF Owlink, it should have faster responeses to queries. Renan will research this. It will probably mean reloading the PingER data so is a lot of work, hopefully this will improve performance. Before the rebuild he will make the fixes and provide a new WAR for us to load on pingerlod.slac.stanford.edu. He is also working on documentation (he has finished the ontology and has a nice interactive tool for visualizing it, since the ontology is the core of the data model of our semantic solution, this will be very helpful for anyone who uses our system, both a developer of the system and a possible user) and his thesis. Bebo pointed out that to get publicity and for people to know about the data, we will need to add pingerlod to lod.org.
Things he will soon do regarding documentation:
- A task/process flow writing all java classes involved on all those batch jobs;
- A Javadoc <http://www.oracle.com/technetwork/java/javase/documentation/index-jsp-135444.html> which will explain all classes and how they are used.
For the Linked Open Data / RDF which is in pre-alpha days, you can go to http://pingerlod.slac.stanford.edu. As can be seen this page is not ready for prime time. However the demos work as long as one carefully elects what to look at:
- Click on Visualizations, there are two choices:
- Multiple Network Metrics: Click on the image: gives a form, choose from Node pinger.slac.stanford.edu pinging to www.ihep.ac.cn, time parameters yearly, 2006 2012, metrics throughput, Average RTT Packet loss and display format Plot graph, then click on submit. In a few seconds time series graph should come up. Mouse over to see details of values at each x value (year).
- A mashup of network metrics x university metrics Click on image: gives another form, pinging from pinger.slac.stanford.edu, School metric number of students, time metric years 2006 2012, display format plot graph, click on submit. Longer wait, after about 35 seconds a google map should show up. Click on "Click for help." Area of dots = number of students, darkness of dots = throughput (lighter is better), inscribing circle color gives university type (public, private etc.) Click on circle for information on university etc.
- Renan will be working on providing documentation on the programs, in particular the install guide for the repository and web site etc. This will assist the person who takes this over.
Renan is using OWLIM as RDF Repository. He is using an evaluation version right now. Renan looked into the price for OWLIM (that excellent RDF Database Management System he told us about). It would cost 1200EUR minimum (~ 1620 USD, according to Google's rate for today) for a one time eternal license. It seems too expensive. No wonder it is so good. Anyhow, he heard about a different free alternative. Just not sure how good it would be for our PingER data. He will try it out and evaluate. He will also get a new evaluation of the free OWLIM lite.
He has also made some modifications on the ontology of the project (under supervision of his professor in Rio) hence he will have to modify the code to load the data accordingly.
Raspberry Pi
A quick comparison of the performance of the two hosts (raspberry pi and regular UNIMAS host) without statistical quantification is available at https://confluence.slac.stanford.edu/display/IEPM/Comparison+of+PinGER+RTTs+from+UNIMAS+monitors+N4+and+RASPBERRY. A page has been created to compare the hardware spec between the pinger.unimas.my node (Intel architecture) and the pinger2.unimas.my node (Raspberry Pi ARM architecture), available from the unimas pinger website at http://pinger.unimas.my/pinger/hardware.php. There is a link to hardware.php in the Comparison+of+PinGER+RTTs+from+UNIMAS+monitors+N4+and+RASPBERRY web page.
Follow up from workshop
- Hossein Javedani of UTM is interested in anomalous event detection with PingER data. Information on this is available at https://confluence.slac.stanford.edu/display/IEPM/Event+Detection. We have sent him a couple of papers and how to access the PingER data. Hossein and Badrul have been put in contact. Is there an update Badrul?
The Next step in funding is to go for bigger research funding, such as LRGS or eScience. Such proposals must lead to publications in high quality journals. They will need an infrastructure such as the one we are building. We can use the upcoming workshop (1 specific session) to brainstorm and come up with such proposal. We need to do some groundwork before that as well. Johari will take the lead in putting together 1/2 page descriptions of the potential research projects.
- Need to identify a few key areas of research related to PingER Malaysia Initiative and this can be shared/publicized through the website. These might include using the infrastructure and data for: anomaly detection; correlation of performance across multiple routes; and for GeoLocation. Future projects as Les listed in Confluence herehttps://confluence.slac.stanford.edu/display/IEPM/Future+Projects can also be a good start and also Bebo's suggestion.
- Need to synchronize and share research proposals so as not to duplicate research works. how to share? Maybe not through the website, or maybe can create a member only section of the website to share sensitive data such as research proposal?
Anjum suggested Saqib, Badrul and Johari put together a paper on user experiences with using the Internet in Malaysia as seen from Malaysian universities. In particular round trip time, losses, jitter, reliability, routing/peering, in particular anomalies, and the impact on VoIP, throughput etc. It would be good to engage someone from MYREN.
Potential projects
Future meeting - Les
Next meeting Wednesday January 22nd 2014 8:00pm Pacific Standard Time, Thursday January 23rd 2014 9:00am Pakistan time, Thursday January 23rd, 2014 noon Malaysian time, Thursday 23rd January, 2014 02:00am Rio Standard Time.