Time & date
Wednesday 4 December 2013 8:00pm Pacific Standard Time, Thursday December 5 2013 9:00am Pakistan time, Thursday 5 December 2013 noon Malaysian time, Thursday 5 December October 2013 01:00am Rio Standard Time.
Attendees
Invitees:
Anjum, +Kashif, Raja+, Johari+, Nara, Abdullah-, Badrul-, Ridzuan-, Hanan, Saqib+, Adib-, Les+, Renan+, Bebo+
+ Confirmed attendance
- Responded but Unable to attend:
Actual attendees:
Anjum, Kashif, Raja, Johari, Saqib, Renan, Bebo, Les, Hassaan Khaliq.
Administration
- Ridzuan has joined. Welcome! Badrul has added him to the pinger-my mailing list, and Johari has added him to the contacts coordinate list of team members.
- Do we need to add anyone else to the weekly meetings, e.g. Hossein Javedani of UTM. Saqib has tried to contact Hossein 3 times. He will try again.
- Anjum will be spending a year at UM starting January. Hasaan Khaliq will take over the leadership of the PingER project at NUST. Hassaan was on the call. Hassaan will send his coordinates to Johari who will update the contacts and email list. Welcome Hassaan!
- Johari proposes that UTM host a PingER workshop in 2014, maybe associated with another conference. Anjum will be in Malaysia then. Saqib will inform Hanan.
- Abdullah from UM may visit SLAC in the upcoming year.
Renan
Renan is using OWLIM as RDF Repository. He is using an evaluation version right now. Renan looked into the price for OWLIM (that excellent RDF Database Management System he told us about). It would cost 1200EUR minimum (~ 1620 USD, according to Google's rate for today) for a one time eternal license. It seems too expensive. No wonder it is so good. Anyhow, he heard about a different free alternative. Just not sure how good it would be for our PingER data. He will try it out and evaluate. He will also get a new evaluation of the free OWLIM lite.
He has also made some modifications on the ontology of the project (under supervision of my professor in Rio) hence he will have to modify the code to load the data accordingly.
He is focusing on his thesis paper. He should present his work on Dec 11. Then he will need to work on the presentation, including the website design and providing more info for the web page (pingerlod.slac.stanford.edu). He is producing good written material about PingER LOD, just wish it were in English already. He will have a hard time translating everything; but it is necessary since he did not provide good documentation about our project.
Renan will provide a new pingerlod war file on Monday Dec 9th. He wants to use this for his thesis defense. Les will contact Tony Johnson to make sure there is no problem getting the war file installed.
After the presentation he will be able work on the code (this has higher priority) and then on the RDF storage alternatives. He will perform more experiments with OWLIM Lite and Open Link Virtuoso, both free of charges. Need to verify if they are good enough for our 50M+ triples.
He has also made some modifications on the ontology of the project (under supervision of his professor in Rio) hence he will have to modify the code to load the data accordingly.
UM
Ridzuan reports his current progress is that he has considered using Apache Mahout (open source machine learning for big data) for the clustering on the Pinger data. He is awaiting Renan decision on final implementation of RDF storage to be adopted. Then, he can start experiment with the new data structure. Renan should he wait or try with the old interfaces (pingtable.pl)? Renan recommends that Ridzuan wait for a week after his thesis defense on Dec 11, hopefully Renan will have something by then. Ridzuan, how did your tests with pingtable.pl go?
Strange minimum RTTs between UM and QIUP and Allianze
- The RTT between UM and UNIMAS took a dramatic leap between 10/24/2013 and 10/27/2013 (following an outage from ~ 35ms to 106ms. Badrul reports their carrier changed.
- There was a further outage from 11/11/2013 to 11/16/2013 after which the RTT dropped to ~ 70ms
Also DATAKL, EMERGE, MAHSA appear to have very indirect routes from UM (i.e. their RTT is much longer than one would expect knowing the distance between the sites).
- Also UM has anomalous RTTs to allianzecollege.edu and QIUP suggesting that these 2 sites are not where they say they are (i.e. they are closer to UM). Should we remove these from monitoring, or can someone tell us where they are really located?
- Badrul has an undergraduate student (Abdulrahim Haroun Ali) looking at such anomalies in ping measurements. They are currently working on it and plan to have a paper by January 2014. They will need help to vet the paper later. Any Progress?
UNIMAS
Raspberry Pi is still in the private network, have not moved the unit to the computer centre. Facing another problem, power supply is loose, very susceptible to power trip if unit is moved by accident, etc. Found a better way to secure the power supply (which is a micro usb connection with uk type plug), Johari has duct taped it into place. It will be returned to computer center by Monday 9th December
Johari is working on a tool to enable synchronizing Malaysian monitors so they all have the same HostList (like the BeaconList). One will enter the new hosts into a web page from which is generated the HostList XML that is fed into the pinger.xml configuration file. There will also be weekly updates. Given that Saqib has generated a set of SE Asian hosts for all Malaysian monitors to make measurements to, this means that UM and UNIMAS will need to update their HostLists. So such a tools will be very useful and timely.
Johari has also installed webmin in both nodes and start collecting performance data such as cpu utilization and network utilization which can be compared between the two nodes. need to find way to embed the graph generated by webmin to be displayed in the website for real time monitoring.
The traceroute server at http://pinger2.unimas.my/cgi-bin/traceroute.pl has the same problem as before. They know (sort of) the problem but haven't got the chance to rectify it (mapping NAT address, needs to be added). There is no progress 12/4/2013.
Johari has a student who is currently finalizing the proposal (it will be submitted next week) in order to officially apply as Master by research student, target to get him officially by December. Currently he is looking at different correlation technique for the project. It was submitted and they plan to hear of acceptance in early January. He is looking at anomaly detection to provide alarms.
UTM
Saqib has finished his thesis. He is applying for a Postgraduate Research assistant-ship at UTM where he plans to work on PingER. He expects to hear the result in December.
Hossein Javedani of UTM was interested in anomaly detection in PingER data. We have sent him a couple of papers and how to access the PingER data. Saqib requested him to join the skype meetings. Waiting for his reply.
Saqib is working with the IT department in UTM to solve the problems of delays in traceroute. They have a new ISP. Now the pinger server is working with single static IP (161.139.68.188) both for intranet and internet. Previous internet IP 161.139.146.158 is removed.
- No load balancing is involved as UTM is working with single ISP.
- Still the problem of the delay in traceroute is not solved. It appears to be due to the DNS lookup delay.
Saqib emailed a list of Potential PingER remote nodes in S. E Asia. We will need to get all Malaysian Pinger monitors to add them to their HostLists. I have started a stub for a SE Asian Case Study at SE Asian Case Study, and within, there is a link to the spreadsheet.
He also investigated Allianze and QIUP concerning their higher values of Directivity.
Allianze from UM has a directivity > 2 for a min-RTT of ~ 1.2ms.
Distance as crow flies between pinger.fsktm.um.edu.my and www.allianzeunicollege.edu.my = 298.4 km. or a minimum RTT (as constrained by speed of light in fibre) = 2.984 msec.
Maybe it has a proxy closer to Kuala Lumpur, or
The lat long of Allianze is badly wrong. Currently it is believed to be at:
Title MY.ALLIANZEUNICOLLEGE.EDU.N1
Nodename www.allianzeunicollege.edu.my
Full Name Allianze University College Of Medical Sciences
Link http://www.allianzeunicollege.edu.my
Type remote
Address Kepala Batas, Penang
Country Malaysia
Region S.E. Asia
Latitude 5.5156
Longitude 100.4393
QIUP from UM has directivity > 1 for a min_RTT ~ 1.3ms or a minimum
Distance as crow flies between pinger.fsktm.um.edu.my and www.allianzeunicollege.edu.my = 173.8km or a minimum RTT (as constrained by speed of light in fibre) = 1.738 msec. So again:
maybe it has a proxy closer to Kuala Lumpur, or
The lat long of QIUP is badly wrong. Currently it is believed to be at:
Title MY.QIUP,EDU.N1
Nodename www.qiup.edu.my
Full Name Quset International University
Type remote
Address No.227, Plaza Teh Teng Seng (Level 2), Jalan Raja Permaisuri Bainun, 30250 Ipoh Perak Darul Ridzuan
Country Malaysia
Region S.E. Asia
Latitude 4.5831
Longitude 101.0958
Saqib recommends that we disable these hosts. The tool Johari is working on should assist in synchronizing adding these for the PingER Malaysian monitoring host. Les has disabled them in the SLAC PingER monitor.
UUM
Dec 4, 2013: The director of computer centre has agreed to support this collaboration and already appointed one staff to be with us during the configuration. Next step, Adib needs to discuss with Prof. Suhaidi to dedicate one PC OR get support from Badrul grant to buy one PC for this purpose or have a bake sale and buy a Raspberry Pi. Adib will do his best to setup PingER monitoring host at UUM before the end of this year.
Regarding the meeting, Adib will attend the upcoming meetings once UUM pinger host is up.
NUST
At the Connect Asia Pacific Summit in Bangkok 2 weeks ago and seeing the project "Mapping the pan Asia Pacific information Superhighway and closing gaps in infrastructure connectivity" Shahryar found that very much related to the work in the PingER project. So Shahryar sent email to a UN agency for a possible collaboration with them on PingER project.
Kashif reports that several of the Pakistani TULIP landmarks had problems.
A) Raja did some changes in the reflector.cgi file
B) There was an Error of “Permission Denied”
Raja sent Nust the list of 10 landmarks that were generating this error while accessing “traceroute.pl” file along with the command of “chcon” to fix it.
This command basically Changes the security context of the FILE.
/var/www/cgi-bin
Chcon –t httpd_unconfined_script_exce_t traceroute.pl
After executing this command we need to restart the httpd service to make it effective.
Nodes | Time | Remarks | |
1 | 12:05 5th Nov | Done | |
2 | 12:13 5th Nov | Done | |
3 | 13:30 5th Nov | Done | |
4 | 16:10 5th Nov | Done | |
5 | 16:17 5th Nov | Done | |
6 | 6th Nov | Done | |
7 | 4 Dec | Done | |
8 | 3 Dec | Done | |
9 | 4 Dec | Done | |
10 | 3 Dec | Done |
With all that we have now 27 Pakistani landmarks which are working fine and 4 will be added in couple of days.
Raja checked the response of currently disabled landmarks and the following appear were giving socket permission errors (these are the ones that weren't problematic before). Kashif reports they have been updated and are working now.
UPDATE: We now have 31 working landmarks in Pakistan, of these 24 are currently active and the remaining 7 are disabled since they have an active landmark within 1km distance from them. This is being carried out automatically using tulip-dup.pl script running every night from trscrontab.
We have been unable to gather PingER data from the following nodes for at least a month. Email sent to Kashif and Joun 11/30/2013:
airuniversity.seecs.edu.pk, This is an active TULIP landmark, so maybe there was a PingER script installation issue. It is fixed now
buitms.seecs.edu.pk. This is an inactive TULIP landmark, but we have another working landmark in Quetta. New system required will send in a week starting Dec 3
duhs.seecs.edu.pk.This is an inactive TULIP landmark, but we have 5 working landmarks in Karachi. CENTOS OS required for re-installation so sending a CD through Courier Dec 4th.
pinger.iba-suk.edu.pk. This is an inactive TULIP landmark in Sukkur, there are no others in the city. IP change in progress.
pingerisl-qau.pern.edu.pk. This is an inactive TULIP landmark, but we have 8 working landmarks in Islamabad. Contact person left so we have no access to the machine or an admin.
pingermtn.pern.edu.pk. This is an inactive TULIP landmark in Multan and the only one, would be nice if we could get this working. Cable problem, we allow them to purchase new and we will re-imburse. Hopefully up in a week.
pingerpwr.pern.edu.pk. This is an inactive TULIP landmark, but we have another working landmark in Peshawar. IP change in progress.
quest.seecs.edu.pk. This is an inactive TULIP landmark in Nawabshah and the only one. IP address is being used elsewhere for some time.
sau.seecs.edu.pk. This is an inactive TULIP landmark in Tandojam and the only one. Concerned person is on leave.
uaf.seecs.edu.pk. This is an inactive TULIP landmark, but we have another working landmark in Faisalabad. Visit required for trouble shooting. Joun and Kashif plan to visit next week.
Anjum believes he can add two more PingER hosts, one at Bahawalpur the other at Sahiwal. These are in central Pakistan towards the Eastern border and so should help with providing TULIP landmarks
TULIP - Raja
The new beta test TULIP site is up and running and is at http://tulip.slac.stanford.edu. Has anybody outside SLAC tried it? Besides the various ping methods, Raja has added information on the target's AS, DNS LOC information if available, and other estimates such as GeoIPTools, GeoPlugin.
We are looking to make the code open source and available via github.
Raja has made lots of measurements and created cumulative probability distribution functions of how the new TULIP with adapting alpha (in distance = Alpha*100(km/ms)*min_RTT(ms)) according to region and min_RTT compares to databases and Constraint Based Geolocation and Single Ping etc.
Raja is working on the impact of landmark density on accuracy. He plans to have this ready in a few days.
Raja is working on a paper.
To first order (due to the number of landmarks available) it only works in N. America, Europe and Pakistan. Even then it is only accurate to a hundred or so km. It also will not work for targets that do not respond to pings or are connected via geo-stationary satellites. We still need to make measurements to lots of targets so we can quantify the statistics here. Its main use at the moment maybe to find roughly the location, i.e. region/country/state, a target is in. This is particularly useful for proxies and for routers (the latter are typically mis-found by GeoIPtools to be in the corporate HQ of the owner (e.g. Berkeley for ESnet routers). It would be really valuable if router owners provided DNS LOC records filled out. If enough did this, we could add this to the balloons estimating the location of the target.
As Raja's work on TULIP begins to go into testing he will be engaging more with Renan and Linked Open Data. Raja has sent email to Renan and has a list of things from Renan that Raja should look at.
PingER at SLAC
Working with Bangladesh to set up a monitoring site at Jahangirnagar University Dhakar. Letter of invitation has been sent. Les requested an update from Bangladesh 8/13/2013. They responded that they are having a delay getting an IP address. M Wahid (of UM) has sent email to Faziul Karim of Jahangirnagar University requesting an update (9/17/2013, 11/2/2013, 11/30/2013)
Les will try again with UFRJ, making sure Renan is on the email so Renan can go beat on the door of the UFRJ contact.
No word from the people below. I am giving up on them, unless Bebo thinks it is worth continuing to try.
- We are also working with MIMOS (see https://confluence.slac.stanford.edu/display/IEPM/20130810+Meeting+with+Dickson+Lukose+of+MIMOS) to get hosts and landmarks. They are in the midst of a discussion. Hope to have something to report by September 8th. Sent email reminders 9/17/2013, 9/27/2013, 11/2/2013. Bebo agrees we give up, it is not Dickson's main field of study.
- Zakaria of Zayed university in Dubai copied Prof. Wathiq Mansoor of the American University in Dubai. Wathiq responded with interest, Les sent him information (basically http://www-iepm.slac.stanford.edu/pinger/letters/invite-monitor.doc). However the site does not appear to respond to pings. Les has not heard anything back from Wathig after informing him of the ping situation. Reminder sent 11/2/2013. Bebo will follow up on this.
Old Items
Renan finished the new pingerlod web site. The new thing is that it should be much easier now to modify the info texts. What Renan did was to put the texts into a separate file. The new version has been loaded on the server and some text added to describe how to use the map. However there is a bug that prevents it from executing the map. Renan reports that the bugs should be easy to fix. He has talked to his professor who suggested trying RDF Owlink, it should have faster responeses to queries. Renan will research this. It will probably mean reloading the PingER data so is a lot of work, hopefully this will improve performance. Before the rebuild he will make the fixes and provide a new WAR for us to load on pingerlod.slac.stanford.edu. He is also working on documentation (he has finished the ontology and has a nice interactive tool for visualizing it, since the ontology is the core of the data model of our semantic solution, this will be very helpful for anyone who uses our system, both a developer of the system and a possible user) and his thesis. Bebo pointed out that to get publicity and for people to know about the data, we will need to add pingerlod to lod.org.
Things he will soon do regarding documentation:
- A task/process flow writing all java classes involved on all those batch jobs;
- A Javadoc <http://www.oracle.com/technetwork/java/javase/documentation/index-jsp-135444.html> which will explain all classes and how they are used.
For the Linked Open Data / RDF which is in pre-alpha days, you can go to http://pingerlod.slac.stanford.edu. As can be seen this page is not ready for prime time. However the demos work as long as one carefully elects what to look at:
- Click on Visualizations, there are two choices:
- Multiple Network Metrics: Click on the image: gives a form, choose from Node pinger.slac.stanford.edu pinging to www.ihep.ac.cn, time parameters yearly, 2006 2012, metrics throughput, Average RTT Packet loss and display format Plot graph, then click on submit. In a few seconds time series graph should come up. Mouse over to see details of values at each x value (year).
- A mashup of network metrics x university metrics Click on image: gives another form, pinging from pinger.slac.stanford.edu, School metric number of students, time metric years 2006 2012, display format plot graph, click on submit. Longer wait, after about 35 seconds a google map should show up. Click on "Click for help." Area of dots = number of students, darkness of dots = throughput (lighter is better), inscribing circle color gives university type (public, private etc.) Click on circle for information on university etc.
- Renan will be working on providing documentation on the programs, in particular the install guide for the repository and web site etc. This will assist the person who takes this over.
A quick comparison of the performance of the two hosts (raspberry pi and regular UNIMAS host) without statistical quantification is available at https://confluence.slac.stanford.edu/display/IEPM/Comparison+of+PinGER+RTTs+from+UNIMAS+monitors+N4+and+RASPBERRY. A page has been created to compare the hardware spec between the pinger.unimas.my node (Intel architecture) and the pinger2.unimas.my node (Raspberry Pi ARM architecture), available from the unimas pinger website at http://pinger.unimas.my/pinger/hardware.php. There is a link to hardware.php in the Comparison+of+PinGER+RTTs+from+UNIMAS+monitors+N4+and+RASPBERRY web page.
Follow up from workshop
- Hossein Javedani of UTM is interested in anomalous event detection with PingER data. Information on this is available at https://confluence.slac.stanford.edu/display/IEPM/Event+Detection. We have sent him a couple of papers and how to access the PingER data. Hossein and Badrul have been put in contact. Is there an update Badrul?
The Next step in funding is to go for bigger research funding, such as LRGS or eScience. Such proposals must lead to publications in high quality journals. They will need an infrastructure such as the one we are building. We can use the upcoming workshop (1 specific session) to brainstorm and come up with such proposal. We need to do some groundwork before that as well. Johari will take the lead in putting together 1/2 page descriptions of the potential research projects.
- Need to identify a few key areas of research related to PingER Malaysia Initiative and this can be shared/publicized through the website. These might include using the infrastructure and data for: anomaly detection; correlation of performance across multiple routes; and for GeoLocation. Future projects as Les listed in Confluence herehttps://confluence.slac.stanford.edu/display/IEPM/Future+Projects can also be a good start and also Bebo's suggestion.
- Need to synchronize and share research proposals so as not to duplicate research works. how to share? Maybe not through the website, or maybe can create a member only section of the website to share sensitive data such as research proposal?
Anjum mentioned a paper on Evaluation of IP Geolocation Algorithms on PingER and PlanetLab Infrastructures, by Fida Gilani et. al. submitted to IEEE INFOCOM 2011. This could be updated with new data and maybe submitted to a different venue. Anjum forwarded a copy of the paper to Johari, Abdullah and Hanan with copies to the team. The idea is to see whether there is interest at UTM, UM or UNIMAS.
Anjum suggested Saqib, Badrul and Johari put together a paper on user experiences with using the Internet in Malaysia as seen from Malaysian universities. In particular round trip time, losses, jitter, reliability, routing/peering, in particular anomalies, and the impact on VoIP, throughput etc. It would be good to engage someone from MYREN.
There was interest in TULIP from Mridul Jain <mridul@yahoo-inc.com>and the Senior Architect in Yahoo! R&D Software Dev in Bangalore, India. He pointed to an interesting paper on Geolocation using CBG and then finding hosts in the area and getting their area codes and using a virtual landmarks making traceroutes. The paper is at https://www.usenix.org/legacy/event/nsdi11/tech/full_papers/Wang_Yong.pdf. He wants something to use in production, however I do not think we are ready.
Potential projects
Future meeting - Les
Next meeting Wednesday January 8th 2014 8:00pm Pacific Standard Time, Thursday January 9th 2014 9:00am Pakistan time, Thursday January 9th, 2014 noon Malaysian time, Thursday 9th January, 2014 02:00am Rio Standard Time.