Time & date

Wednesday 6 November  2013 8:00pm Pacific Standard Time, Thursday November 7  2013 9:00am Pakistan time, Thursday 7 November  2013 noon Malaysian time, Thursday 7 NovemberOctober 2013 01:00am Rio Standard Time.

Attendees

Invitees: Anjum-, Kashif (1st 45 mins), Raja+, Johari+, Nara, Abdullah, Badrul+, Hanan, Saqib+, Adib, Les+, Renan, Bebo

+ Confirmed attendance

- Responded but  Unable to attend

  • Actual attendees: Kashig (bit microphone did not work), Raja, Johari, Badrul, Saqib, Les

Administration

  • Ridzuan has joined, he was on the phone with Badrul. Badrul wull add him to pinger-my, and Johari will add him to the contacts.
  • Do we need to add anyone else to the weekly meetings, e.g. Hossein Javedani of UTM (or maybe Saqib or Hanan can cover for him).

Renan

Renan is focusing a lot on his thesis paper now. He is using PingER LOD as his example so, yes, he is  working on our project. Right now he is mainly writing and giving lectures at UFRJ about web science and Linked Open Data. And he is always talking about PingER so lots of Brazilians know about the project now!  Most of the stuff he is writing on in his paper can be used as documentation for PingER LOD so it's good. The only down side is that he was forced to write everything in Portuguese so he  will have to translate everything later.

Specifically about the project, he is using OWLIM as RDF Repository but as said in previous emails, it's paid. He is using an evaluation version right now. After the presentation of his work, he will have more time to see what's better for us to use in PingER LOD project.

Renan  finished the new pingerlod web site. The new thing is that it should be much easier now to modify the info texts. What Renan did was to put the texts into a separate file. The new version has been loaded on the server and some text added to describe how to use the map. However there is a bug that prevents it from executing the map. Renan reports that the bugs should be easy to fix. He has talked to his professor who suggested trying RDF Owlink, it should have faster responeses to queries. Renan will research this.  It will probably mean reloading the PingER data so is a lot of work, hopefully this will improve performance. Before the rebuild he will make the fixes and provide a new WAR for us to load on pingerlod.slac.stanford.edu. He is also working on documentation (he has finished the ontology and has a nice interactive tool for visualizing it, since the ontology is the core of the data model of our semantic solution, this will be very helpful for anyone who uses our system, both a developer of the system and a possible user) and his thesis. Bebo pointed out that to get publicity and for people to know about the data, we will need to add pingerlod to lod.org.

Things he will soon do regarding documentation:

  1. A task/process flow writing all java classes involved on all those batch jobs;
  2. A Javadoc <http://www.oracle.com/technetwork/java/javase/documentation/index-jsp-135444.html> which will explain all classes and how they are used.

UNIMAS

The pinger2 (Raspberry Pi) unit is down and Johari took it back to the faculty for troubleshooting. He found a corrupted sd card, unable to retrieve data from previous installation. He decided to switch to a larger capacity, 16GB SDHC CLass 4 card. Currently running in private network for at least another 1 week to test for stability before moving the unit back to the computer centre for redeployment

A quick comparison of the performance of the two hosts without statistical quantification is available at https://confluence.slac.stanford.edu/display/IEPM/Comparison+of+PinGER+RTTs+from+UNIMAS+monitors+N4+and+RASPBERRY.  A page has been created to compare the hardware spec between the pinger.unimas.my node (Intel architecture) and the pinger2.unimas.my node (Raspberry Pi ARM architecture), available from the unimas pinger website at http://pinger.unimas.my/pinger/hardware.php. There is a link to hardware.php in the Comparison+of+PinGER+RTTs+from+UNIMAS+monitors+N4+and+RASPBERRY web page.

Johari has also installed webmin in both nodes and start collecting performance data such as cpu utilization and network utilization which can be compared between the two nodes. need to find way to embed the graph generated by webmin to be displayed in the website for real time monitoring.

The traceroute server at http://pinger2.unimas.my/cgi-bin/traceroute.pl  has the same problem as before. They know (sort of) the problem but haven't got the chance to rectify it (mapping NAT address, needs to be added). The ping server landmark is working.

Custom iso: tried using remastersys tool to create custom iso, able to generate the custom iso but unable to boot from the iso due to kernel panic. Still troubleshooting the issue and also looking for other tool for creating custom iso.

Johari is currently creating a shell script to automate the installation of pinger package in Ubuntu/Linux distro. He is using SLAC repo version 2.3 with a virtual box and ubuntu server 13.04 for testing purpose, and the script seems to be working fine. There was a problem when executing the pinger2.pl script, that the script could not find the lynx installed in the system. which lynx returns /usr/bin/lynx but the error is pointing to /usr/local/bin/lynx. He fixed this by downloading and using the current pinger.xml file from pinger.unimas.my replacing the pinger.xml file from the pinger package, and it works, the pinger2.pl script manage to update the beacons list and run the pinger program.  The problem is SLAC stores lynx in /usr/local/bin, whereas most sites use /usr/bin.

Johari has a student who  is currently finalizing the proposal (it will be submitted next week) in order to officially apply as Master by research student, target to get him officially by December. Currently he is looking at different correlation technique for the project.

UTM

Saqib has finished his thesis. He is applying for a Postgraduate Research assistant-ship at UTM where he plans to work on PingER. He expects to hear the result in December.

Hossein Javedani of UTM who was interested in anomaly detection in PingER data. We have sent him a couple of papers and how to access the PingER data.  Saqib requested him to join the skype meetings. Waiting for his reply 11/5/2013.

Saqib is working with the IT department in UTM to solve the problems of delays in traceroute.  They have a new ISP.  Now the pinger server is working with single static IP (161.139.68.188) both for intranet and internet.   Previous internet IP 161.139.146.158 is removed.  

  • No load balancing is involved as UTM is working with single ISP. 
  • Still the problem of the delay in traceroute is not solved. It appears to be due to the DNS lookup delay.

Saqib is searching for pingable universities in SE Asia to incorporate them as remote sites in UTM PingER server.

He is also investigating Allianze and  QIUP about their higher values of Directivity. Johari believes these hosts are not in Malaysia. However:

  • Looking at the PingER results for QIUP seen from UM & UTM we see it is ~ 2ms from from UM and 15ms from UTM. Tulip has problems with it in that target is not responding to tier 1 landmarks.
  • Allianze is 2 ms from UM and 14ms from UTM. The TULIP CBG algorithm did not converge (does not have 3 landmarks overlapping).
  • This still does not explain why the directivity for UM for these two targets is > 1, e.g. 
    • Allianze from UM has a directivity > 2 for a min-RTT of ~ 1.2ms.
    • Distance as crow flies between pinger.fsktm.um.edu.my and www.allianzeunicollege.edu.my = 298.4 km. or a minimum RTT (as constrained by speed of light in fibre) = 2.984 msec.
    • Maybe they have a proxy closer to Kuala Lumpur

UM

  • Badrul has an undergraduate student (Abdulrahim Haroun Ali) looking at anomalies in ping measurements. Les and the undergraduate student have corresponded concerning how to access the PingER data Sept 25-27.  They are currently working on it and plan to have a paper by January 2014. They will need help to vet the paper later.
  • MD NOR RIDZUAN BIN DAUD of UM is interested in analytics with Big Data for PingER.  Renan has sent documentation and is in contact with Md. Md plans to to look at analytics of clustering correlation in PingER data. He will get the data from pingtable.pl. He will try a small amount of data first and extend to more data later. Seehttps://confluence.slac.stanford.edu/display/IEPM/PingER+Big+Data+Analytics+proposal. 

NUST

Johari asked whether the NUST PingER data archive goes back to 1998. Perhaps Anjum or Kashif could assist with the answer to this.

TULIP - Raja

The new beta test TULIP site is up and running. It is at: http://www.slac.stanford.edu/comp/net/tulip/. Raja has made great progress in speeding it up. The main problem was that the parallel tasks were not working right. He has incorporated the tiering and optimized the choice of tier 0 landmarks. He has also got the laundering working so now we can properly disable landmarks that are not responding and add disabled landmarks that are responding. This, together with looking at the perfSONAR site for landmarks,  has increased the number of landmarks. There are now 55 PingER active landmarks, 59 perfSONAR and 79 PlanetLabs active landmarks. 

The GUI is working. Raja has added balloons for both the CBG and ACBG estimates as well as GeoIPTools and Networld estimates where available. He has also added the nearest landmarks, the circles estimated from the minimum RTTs  and the convex hull obtained by ACBG.  We are looking at adding a tier 0 host for S. E. Asia and another for Australia.

We are also looking at what to do if there is no convergence, e.g. just show the circles. We are also looking at adding tier 0s for S.E. Asia and Australia.

We (the PingER team) need the team to try it out both to find silly errors like missing or bad  URLs, improve the navigation, and how it works for various sites with known locations. Please do this and report on findings. There is a from for reporting results, it is under Give Feedback after running the geolocation.

Also any other ideas on how to find the location (lat/long) of sites would be welcome. Plus any help in adding to the number of working landmarks at known locations would be invaluable.

To first order (due to the number of landmarks available) it only works in N. America, Europe and Pakistan. Even then it is only accurate to a hundred or so km. It also will not work for targets that do not respond to pings or are connected via geo-stationary satellites. We still need to make measurements to lots of targets so we can quantify the statistics here. Its main use at the moment maybe to find roughly the location, i.e. region/country/state,  a target is in. This is particularly useful for proxies and for routers (the latter are typically mis-found by GeoIPtools to be in the corporate HQ of the owner (e.g. Berkeley for ESnet routers). It would be really valuable if router owners provided DNS LOC records filled out.  If enough did this, we could add this to the balloons estimating the location of the target.

Asad has identified about 10 Pakistani landmarks that are misconfigured, plus a probable way to fix then using the chcon command (Chcon –t httpd_unconfined_script_exce_t traceroute.pl). This command basically changes the security context of the FILE. After executing this command we need to restart the httpd service to make it effective. So, out of 10 landmarks they (Joun/Kashif) have fixed 6 (we now have 26 working Pakistani landmarks) and they are working fine now. For rest of 4, they are in contact with the admins and hopefully it will be fixed in 1-2 days.

As Asad's work on TULIP begins to go into testing he will be engaging more with Renan and Linked Open Data. Asad will send email to Renan.

PingER at SLAC 

  • Working with Bangladesh to set up a monitoring site at Jahangirnagar University Dhakar. Letter of invitation has been sent. Les requested an update from Bangladesh 8/13/2013. They responded that they are having a delay getting an IP address. M Wahid (of UM) has sent email to Faziul Karim of Jahangirnagar University requesting an update (9/17/2013, 11/2/2013)
  • We are also working with MIMOS (see https://confluence.slac.stanford.edu/display/IEPM/20130810+Meeting+with+Dickson+Lukose+of+MIMOS) to get hosts and landmarks. They are in the midst of a discussion. Hope to have something to report by September 8th. Sent email reminders 9/17/2013, 9/27/2013, 11/2/2013
  • Adib at UUM is waiting on the computer centre director to appoint one staff to be with us during the installation and configuration. They cannot do it without their supervision to avoid any issue in the future. 9/27/2013, sent reminder 11/2/2013.
  • Zakaria of Zayed university in Dubai copied Prof. Wathiq Mansoor of the American University in Dubai. Wathiq responded with interest, Les sent him information (basically http://www-iepm.slac.stanford.edu/pinger/letters/invite-monitor.doc). However the site does not appear to respond to pings. Les has not heard anything back from Wathig after informing him of the ping situation. Reminder sent 11/2/2013.
Old Items

For the Linked Open Data / RDF which is in pre-alpha days, you can go to http://pingerlod.slac.stanford.edu. As can be seen this page is not ready for prime time. However the demos work as long as one carefully elects what to look at:

  • Click on Visualizations, there are two choices:
    • Multiple Network Metrics: Click on the image: gives a form, choose from Node pinger.slac.stanford.edu pinging to www.ihep.ac.cn, time parameters yearly, 2006 2012, metrics throughput, Average RTT Packet loss and display format Plot graph, then click on submit. In a few seconds time series graph should come up. Mouse over to see details of values at each x value (year).
    • A mashup of network metrics x university metrics Click on image: gives another form, pinging from pinger.slac.stanford.edu, School metric number of students, time metric years 2006 2012, display format plot graph, click on submit. Longer wait, after about 35 seconds a google map should show up. Click on "Click for help." Area of dots = number of students, darkness of dots = throughput (lighter is better), inscribing circle color gives university type (public, private etc.) Click on circle for information on university etc.
  • Renan will be working on providing documentation on the programs, in particular the install guide for the repository and web site etc. This will assist the person who takes this over. 

Follow up from workshop

  • Adib of UUM  also interested in porting PingER to support IPv6.
    • Anjum reported that SEECS started on this. They were thinking of using the IPv6 address, then using it to get the IPv4 address and rewriting the ping measurement with the IPv4 address, then everything in the analysis stays as it was. An alternative might be to get rid of the IP addresses altogether. Anjum reached out to Adib.
  • Hossein Javedani of UTM is interested in anomalous event detection with PingER data. Information on this is available at https://confluence.slac.stanford.edu/display/IEPM/Event+Detection. We have sent him a couple of papers and how to access the PingER data. Hossein and Badrul have been put in contact. Is there an update Badrul?

The Next step in funding is to go for bigger research funding, such as LRGS or eScience. Such proposals must lead to publications in high quality journals. They will need an infrastructure such as the one we are building. We can use the upcoming workshop (1 specific session) to brainstorm and come up with such proposal. We need to do some groundwork before that as well. Johari will take the lead in putting together 1/2 page descriptions of the potential research projects. 

  1. Need to identify a few key areas of research related to PingER Malaysia Initiative and this can be shared/publicized through the website. These might include using the infrastructure and data for: anomaly detection; correlation of performance across multiple routes; and for GeoLocation. Future projects as Les listed in Confluence herehttps://confluence.slac.stanford.edu/display/IEPM/Future+Projects can also be a good start and also Bebo's suggestion. 
  2. Need to synchronize and share research proposals so as not to duplicate research works. how to share? Maybe not through the website, or maybe can create a member only section of the website to share sensitive data such as research proposal?

 Anjum mentioned a paper on Evaluation of IP Geolocation Algorithms on PingER and PlanetLab Infrastructures, by Fida Gilani et. al. submitted to IEEE INFOCOM 2011. This could be updated with new data and maybe submitted to a different venue. Anjum forwarded a copy of the paper to Johari, Abdullah and Hanan with copies to the team. The idea is to see whether there is interest at UTM, UM or UNIMAS. 

Anjum suggested Saqib,  Badrul and Johari put together a paper on user experiences with using the Internet in Malaysia as seen from Malaysian universities. In particular round trip time, losses, jitter, reliability, routing/peering, in particular anomalies, and the impact on VoIP, throughput etc.  It would be good to engage someone from MYREN.

There was interest in TULIP from Mridul Jain <mridul@yahoo-inc.com>and the Senior Architect in Yahoo! R&D Software Dev in Bangalore, India. He pointed to an interesting paper on Geolocation using CBG and then finding hosts in the area and getting their area codes and using a virtual landmarks making traceroutes. The paper is at https://www.usenix.org/legacy/event/nsdi11/tech/full_papers/Wang_Yong.pdf. He wants something to use in production, however I do not think we are ready.

Potential projects

See list of Projects

Future meeting  - Les

Next meeting Wednesday 8 January 2014 8:00pm Pacific Standard Time, Thursday 9 January 2014 9:00am Pakistan time, 9 January 2014 noon Malaysian time, Thursday 9 January 2014 01:00am Rio Standard Time.

Coordinates of team members:

See: http://pinger.unimas.my/pinger/contact.php

  • No labels