20141210 SLAC SEECS and UNIMAS Meeting

Time & date 

Wednesday  Dec 10th,  2014 8:00pm Pacific Standard Time, Thursday Dec 11th 2014 9:00am Pakistan time, Thursday Dec 11th 12:00 noon Malaysian time, Thursday Dec 11th, 2014 2:00am Rio Standard Time.

Attendees

Invitees:

Anjum+, Hassaan Khaliq, Kashif, Raja-,  Samad Riaz, Johari+, Nara, Adnan, Abdullah, Badrul, Ridzuan, Ibrahim, Hanan, Saqib-, Adib, Les+, Renan, Bebo+

+ Confirmed attendance

- Responded but  Unable to attend: 

Actual attendees:

Anjum, Johari, Les, Bebo. Samad got the time mixed up and was too late to attend.

After the last meeting we added Samad at SEECS to the list of invitees.  We have sent him another invite so Les can call Samad on Skype.

Administration

• The workshop 1-866-740-1260, updates from Johari, Anjum, Adib.   It went well there were about 18 attendants mainly students and PhDs. A MYREN technical guy seemed very interested and there has been an exchange of emails.  He is not at the decision level so is probably working with upper management to get approval.  The idea would be that MYREN would install a PingEr monitor at a MYREN hosts and monitor about 20 MYREN hosts at NOCs and other sites. Given the end of year holidays there may be some delay. Joahari will reprompt the  MYREN person in the New Year.
  • There are links to the presentations by Johari and Anjum at PingER Papers and Presentations.
  • Follow up with MYREN.
  • Anjum feels we need to engage the universities then extend to MYREN. Following up with the Malaysia Ministry of education (MoE), to get funding for universities would be a next step. This would come better from Johari than Anjum. Anjum will send Johari details of how this worked in Pakistan (with HEC). 
• Anjum and Raja have been working on a paper on Geolocation as developed for TULIP. Using an exponential relation between the Directivity (Alpha) and RTT for Pakistan the accuracy is ~ 18Km. Now Raja needs to run for Europe and the US. Meanwhile Raja has got a job and has less time to work on this so it is stalled.
• Anjum's contract in Malaysia ends in 50 days. It may be extended or he may pursue a Senior lectureship  elsewhere.

UUM

 As of 10/13/2014 pinger.uum.edu.my was not pingable from SLAC. Adib fixed earlier this week. There was an interesting plateau in the RTTs to UUM from SLAC see below (black means the host was no reachable). Apart from the delay the performance in terms of lack of congestion and loss appears to be the same. 

The route change seems to be after hop 13 and before hop 16.  See http://www-wanmon.slac.stanford.edu/cgi-wrap/traceroutearchive.cgi?from=www-wanmon.slac.stanford.edu&to=pinger.uum.edu.my&date1=2014_09_14&date2=2014_09_22&date3=2014_09_27

Renan

Maria Luiza Campos of UFRJ reports that there are people at UFRJ taking care of the PingER data analytics project at this moment. Maria is now in LOA Trento Italy for 1 year post doc. Adriana Vivacqua, is now in charge of this subproject at CRDB at UFRJ.  There are 11 people on the team: 4 professors1 post doc, 1 doctoral student, 2 masters students and 3 undergraduates.There is a document in which they describe the project proposal with more details. 

Renan should continue their work as soon as Cristiane Ceia begins her BSc dissertation thesis, in which Renan will be her co-advisor (together with Luiza).

UM

Badrul (6/23/2014) is still awaiting hearing from his student (Abdulrahim Haroun Ali who is out of the country) on  the paper on anomalies in PingER measurements  and will update later once the paper ready. For the minute the paper is not ready. No update 8/20/2014, 10/16/2014, 12/10/2014.

Ridzuan and Ibrahim both requested for additional space. We require an infrastructure update for the cloud because the existing storage space is not enough, given a number of other users. The 5 Tera byte SAS along with some blade servers all have minimum 1Gig interfaces. We are in process of purchasing the switch to support the networking. Once available, we shall be able to provide the capacity for one instance of SLAC data. Anjum will provide this over the coming weekend (by 15 Dec 2014)

UNIMAS

Pinger2 (Raspberry Pi) ran  successfully from Sept 2nd to October 23. We have been unable to gather data since. It still responds to pings. The graphs below show the RTTs from SLAC to www, pinger and raspberry pi at UNIMAS. At first glimpse they are very similar. www & pi are probably closer together (more jitter and lower loss)  in RTT performance  than pinger. To quantify one would need to do a frequency analysis and possibly a Kolmogorov/Smirnov test. 

It might be more instructive to look at the data from pinger and raspberry pi to Malaysia since the distances are shorter and the differences may show up better. For Sep-Oct 2014 when there was data measured from both Oct-Nov the averages for 20 paths was 52+-21ms (from pinger.unimas.my to 20 other Malaysian hosts) and 56+-21ms for raspberry pi to 20 other Malayisn hosts.

Traceroute server: Status unsolved. The problem is the same on Pinger2. Johari talked to the network administrator at the centre about this issues and he suggested to talk to the security manager to check whether the firewall is blocking the icmp packer from the traceroute command (to do list). It is possible/probable that they are blocking the default traceroute UDP ports 33434 - 33465. Johari has superuser privs on the machine and tried the traceroute -I to see if ICMP pings work, and -T -p 80 to see if TCP probes (SYN) to port 80 get  through OK.  They do so the next step is to temprarily unblock UDP ports 33434 - 33465 to see if that fixes it.

UTM

After revision the FRGS proposal was submitted to RMC. It was not accepted. We need to update it again in order to fulfill the requirements of the grant.

Saqib has updated the case study and is available in Google drive as a "Shared-PingER" document for review at https://drive.google.com/folderview?id=0B-NEKleLll79ZFNmUnhiVGJ0Nmc&usp=sharing_eid (thanks to Bebo who will notify all of how to access). Further it needs some updates from UNIMAS (on Raspberry  Pi),  UM (on big data) and UUM.

http://pinger.fsksm.utm.my/cgi-bin/traceroute.pl?function=ping works. traceroute problem regarding maximum reachable hops ( i.e. 11 hopes ) is still unresolved.  The traceroute looks as below.

Executing exec(traceroute -m 30 -q 3 134.79.197.197 140)
traceroute to 134.79.197.197 (134.79.197.197), 30 hops max, 140 byte packets
 1  161.139.68.250 (161.139.68.250)  0.974 ms  1.293 ms  1.524 ms
 2  10.110.1.67 (10.110.1.67)  1.135 ms  1.495 ms  1.447 ms
 3  161.139.248.254 (161.139.248.254)  3.468 ms  3.360 ms  3.269 ms
 4  161.139.245.254 (161.139.245.254)  4.889 ms  4.876 ms  4.968 ms
 5  * * *
 6  * * *
 7  203.80.23.153 (203.80.23.153)  17.659 ms  17.623 ms  17.573 ms
 8  203.80.22.137 (203.80.22.137)  16.069 ms  15.098 ms  16.090 ms
 9  203.80.23.242 (203.80.23.242)  12.634 ms  12.791 ms  12.831 ms
10  202.179.249.85 (202.179.249.85)  22.090 ms  21.950 ms  22.124 ms
11  jp-pop-sg-v4.bb.tein3.net (202.179.249.78)  88.761 ms  88.710 ms *
12  * * *
13  * * *

Saqib has root access to  the pinger monitor at UM. However IP address of hop 6 from UM to UNISAM is not pingable. Thus he can't look at  time series of RTT to see if there are good and bad periods.

NUST

We are unable to ping (or get data from)from the following hosts (comments following # sign are updates from Samad)

• buitms.seecs.edu.pk #The person does not wish to continue working with us.
• comsatsswl.seecs.edu.pk
• nuisb.pieas.edu.pk
• nukhimain.seecs.edu.pk
• Pinger.comsats.edu.pk #This node is working now.
• pinger.nca.edu.pk #This node is working now.
• pinger.uettaxila.edu.pk #The node need fresh installation but Samad has issue in visiting the site that's why it is not working from many days.
• pingerisl-fjwu.pern.edu.pk #There is a problem in the link to that node that's why it is not working from a long time. i am in contact with the concern person. hope it will be up in a week.
• pingerkhi.pern.edu.pk
• pinger.lhr.pern.edu.pk
• sau.seecs.edu.pk

The following are pingable but there is no data:

• ns3.pieas.edu.pk
• pingerusindh.edu.pk #The node is working now.
• pingerkhi-cpsp.pern.edu.pk
• www.upesh.edu.pk #Need installation.will be up in a day. The concern person will re install Pinger today according to our commitment. 

PingER at SLAC

The  Burkina Faso workshop was postponed since the government fell, there was civil unrest, the capital's (Ouagadougou) airport closed, there were deaths etc. 

Next meeting:  Wednesday Jan 7th 2015 8:00pm Pacific Standard Time, Thursday Jan 8th  2015  9:00am Pakistan time, Thursday Jan 8th 2015 noon Malaysian time, Thursday  Jan 8th, 2015 02:00am Rio Standard Time.  

Bebo came up with a self funded researcher who is interested in communications. Les needs to review the information.

Old Items

Traceroute from UM to UNIMAS

According to Professor Francis Lee, SingAREN SLIX core router is a key node for international research and education networks – including APAN, GLORIAD, Internet2, and TEIN – and peers directly with Australia’s AARNet and Japan’s NII and NICT networks. The first 100 GbE international connection is likely to be made within the next year as a result of a US funding call for a 100 Gbps research network link to Asia.  PingER may be very valuable for seeing the impact.

Saqib has run the mtr for more than 3 days from UM to UNIMAS, UTM to UM, and UTM to UNIMAS. From UM to UNIMAS, best RTT from 6 to 9, gets down to ~2ms from ~49ms (for the worst case).  However, for hop 10 it stayed at ~42ms as shown in attached figure (UM-UNIMAS). Thus it appears the the MYREN guy is right that there is much congestion at hop 6 and possibly 7-9 for most of the time. Below are the UM-UNIMAS mtr results.

It sounds like the MYREN guy is right  that there is significant congestion at least at hop 6. I wonder if we can get a better handle on this by monitoring hop 6 from UM and seeing the time periods when the congestion occurs. Some one would need to add to pinger.xml <HostList> something like:

 <Host>
<Alarm>
<TimeOfFirstFailure>1410091899</TimeOfFirstFailure>
</Alarm>
<DnsLastChecked>1410982759</DnsLastChecked>
<IP>203.80.23.73</IP>
<Name> te-0-3-0-0.drc96.jaring.my</Name> 
</Host>

It is possible it may not respond to pings which will make this not useful. Check that first. I notice from SLAC:

 249cottrell@pinger:~$ping 203.80.23.73

PING 203.80.23.73 (203.80.23.73) 56(84) bytes of data.
^C
--- 203.80.23.73 ping statistics ---
55 packets transmitted, 0 received, 100% packet loss, time 54850ms

 However: 

 250cottrell@pinger:~$ping te-0-3-0-0.drc96.jaring.my PING te-0-3-0-0.drc96.jaring.my (61.6.51.2) 56(84) bytes of data.
64 bytes from te-0-3-0-0.drc96.jaring.my (61.6.51.2): icmp_seq=1 ttl=242 time=192 ms
64 bytes from te-0-3-0-0.drc96.jaring.my (61.6.51.2): icmp_seq=2 ttl=242 time=194 ms
64 bytes from te-0-3-0-0.drc96.jaring.my (61.6.51.2): icmp_seq=3 ttl=242 time=193 ms ^C
--- te-0-3-0-0.drc96.jaring.my ping statistics ---
4 packets transmitted, 3 received, 25% packet loss, time 3193ms rtt min/avg/max/mdev = 192.982/193.542/194.490/0.763 ms

I.e. the name refers to a different IP address.

Linked Open Data

Renan  finished the new pingerlod web site. The new thing is that it should be much easier now to modify the info texts. What Renan did was to put the texts into a separate file. The new version has been loaded on the server and some text added to describe how to use the map. However there is a bug that prevents it from executing the map. Renan reports that the bugs should be easy to fix. He has talked to his professor who suggested trying RDF Owlink, it should have faster responses to queries. Renan will research this.  It will probably mean reloading the PingER data so is a lot of work, hopefully this will improve performance. Before the rebuild he will make the fixes and provide a new WAR for us to load on pingerlod.slac.stanford.edu. He is also working on documentation (he has finished the ontology and has a nice interactive tool for visualizing it, since the ontology is the core of the data model of our semantic solution, this will be very helpful for anyone who uses our system, both a developer of the system and a possible user) and his thesis. Bebo pointed out that to get publicity and for people to know about the data, we will need to add pingerlod to lod.org.

Things he will soon do regarding documentation:

1. A task/process flow writing all java classes involved on all those batch jobs;
2. A Javadoc <http://www.oracle.com/technetwork/java/javase/documentation/index-jsp-135444.html> which will explain all classes and how they are used.

For the Linked Open Data / RDF which is in pre-alpha days, you can go to http://pingerlod.slac.stanford.edu. As can be seen this page is not ready for prime time. However the demos work as long as one carefully elects what to look at:

• Click on Visualizations, there are two choices:
  • Multiple Network Metrics: Click on the image: gives a form, choose from Node pinger.slac.stanford.edu pinging to www.ihep.ac.cn, time parameters yearly, 2006 2012, metrics throughput, Average RTT Packet loss and display format Plot graph, then click on submit. In a few seconds time series graph should come up. Mouse over to see details of values at each x value (year).
  • A mashup of network metrics x university metrics Click on image: gives another form, pinging from pinger.slac.stanford.edu, School metric number of students, time metric years 2006 2012, display format plot graph, click on submit. Longer wait, after about 35 seconds a google map should show up. Click on "Click for help." Area of dots = number of students, darkness of dots = throughput (lighter is better), inscribing circle color gives university type (public, private etc.) Click on circle for information on university etc.

• Renan will be working on providing documentation on the programs, in particular the install guide for the repository and web site etc. This will assist the person who takes this over. 

Renan is using OWLIM as RDF Repository. He is using an evaluation version right now. Renan looked into the price for OWLIM (that excellent RDF Database Management System he told us about). It would cost 1200EUR minimum  (~ 1620 USD, according to Google's rate for today) for a one time eternal license. It seems too expensive. No wonder it is so good. Anyhow, he heard about a different free alternative. Just not sure how good it would be for our PingER data. He will try it out and evaluate. He will also get a new evaluation of the free OWLIM lite.  

He has also made some modifications on the ontology of the project (under supervision of his professor in Rio) hence he  will have to modify the code to load the data accordingly.

Maria and Renan are advancing in some approaches to deal with PingER data, making it easier to be analyzed and integrated. In particular they have been busy studying and evaluating alternatives, analyzing results from the latest benchmarks on NoSQL (including RDF and graph based storages) database management, distributed processing and mediated  solutions over relational databases, and also other experiments with multidimensional analyses on Linked Data.  The new students involved are now understanding better the scenario and they have been interacting with Renan regularly. 

They have separated the tasks into 2: 

1. Quantitative analysis on PingER data
   1. They want to know how PingER has grown, since 1998 until today and how it might be in the next years. By doing this, we may focus on more suitable technologies that deal with scenarios that have a similar profile with PingER.
      1. Two students are working on this.
         
2. Approaches to handle PingER current data
   1. Conventional approach – Utilization of Cassandra as back-end database to provide easy crossing of parameters to get PingER data.
      1. One student is working on this.
   2. Distributed and parallel approach – Utilization of a data warehouse on top of a distributed file system to provide low latency response to complex queries (like the ones we were not able to do on my previous work). Additionally, how Scientific Workflow Management Systems may help in the ETL process of transforming PingER so it can easily be stored on the data warehouse.
      1. Renan is working on this.
   3. Pure RDF approach – Good ways of modeling and natively storing RDF data.
      1. Maria-Luiza is working on this.
   4. NoSQL approaches – How other NoSQL DBMS may be adequate for PingER multidimensional data.
      1. Two students are evaluating existing NoSQL solutions for multidimensional scenarios (such as PingER)
   5. Key-Value storages for PingER data in RDF
      1. This is Ibrahim’s work.

In the end, they want to compare all these approaches.

NUST

At the Connect Asia Pacific Summit in Bangkok in  January and seeing the  project "Mapping the pan Asia Pacific information Superhighway and closing gaps in infrastructure  connectivity" Shahryar found that very much related to the work in the PingER project. So Shahryar sent email to a UN agency for a possible collaboration with them on PingER project. He has heard nothing so he will write a detailed proposal and then should contact them again. No update 2/5/2014, 3/5/2014.

Tulip

Follow up from workshop

• Hossein Javedani of UTM is interested in anomalous event detection with PingER data. Information on this is available at https://confluence.slac.stanford.edu/display/IEPM/Event+Detection. We have sent him a couple of papers and how to access the PingER data. Hossein and Badrul have been put in contact. Is there an update Badrul?

The Next step in funding is to go for bigger research funding, such as LRGS or eScience. Such proposals must lead to publications in high quality journals. They will need an infrastructure such as the one we are building. We can use the upcoming workshop (1 specific session) to brainstorm and come up with such proposal. We need to do some groundwork before that as well. Johari will take the lead in putting together 1/2 page descriptions of the potential research projects. 

1. Need to identify a few key areas of research related to PingER Malaysia Initiative and this can be shared/publicized through the website. These might include using the infrastructure and data for: anomaly detection; correlation of performance across multiple routes; and for GeoLocation. Future projects as Les listed in Confluence herehttps://confluence.slac.stanford.edu/display/IEPM/Future+Projects can also be a good start and also Bebo's suggestion. 
2. Need to synchronize and share research proposals so as not to duplicate research works. how to share? Maybe not through the website, or maybe can create a member only section of the website to share sensitive data such as research proposal?

Anjum suggested Saqib,  Badrul and Johari put together a paper on user experiences with using the Internet in Malaysia as seen from Malaysian universities. In particular round trip time, losses, jitter, reliability, routing/peering, in particular anomalies, and the impact on VoIP, throughput etc.  It would be good to engage someone from MYREN.

Ibrahim

Ibrahim Abaker  is planning to work on a topic initially entitled " leveraging pingER big data with a modified pingtable for event-correlation and clustering".  Ibrahim has a proposal, see https://confluence.slac.stanford.edu/download/attachments/17162/leveraging+pingER+big+data+with+a+modified+pingtable+for+event-correlation+and+clustering.docx. Ibrahim reports 7/15/2014 "I have spent the last few months trying to understand the concept of big data storage and its retrieval as well as the traditional approach of storing RDF data. I have integrated a single hadoop cluster in our cloud. but for this project we need multiple clusters, which I have already discussed with Dr. Badrul and he will provide me with big storage for the experiment." No Update 8/20/2014.

"I have come up with initial proposed solution model. This model consists of several parts. The upper parts of the Figure below shows the data source, in which PingER data will be convert into RDF format. Then the data pre-processor will take care of converting RDF/XML into N-triples serialization formats using N-triples convertor module. This N-triple file of an RDF graph will be as an input and stores the triples in storage as a key value pair using MapReduce jobs"

Potential projects

See list of Projects

Coordinates of team members:

See: http://pinger.unimas.my/pinger/contact.php