...
- Responded but Unable to attend:
Actual attendees:
- Adib, Johari, Adnan, Les, Bebo and Hafizi Jalil of MYREN.
- Saqib had network problems,
- Anjum had a meeting that ran over.
- Hassaan had a death in the family he will update the NUST hosts information.
Administration
- Hafizi Jalil of MYREN joined the meeting. He introduced himself. He has been at MYREN for 1.5 years. He maintains development years. He maintains developments for the MYREN network. He has PingER installations at Cyberjaya and UNIMAS. He has more machines, currently running perfSONAR at UMP, UTM, UM and another site. This week he will install PingER and the traceroute servers on these machines. A goal would be to compare and contrast the benefits of perfSONAR and PingER.
- Anjum and Raja have been working on a paper on Geolocation as developed for TULIP. Using an exponential relation between the Directivity (Alpha) and RTT for Pakistan the accuracy is ~ 18Km. Now Raja needs to run for Europe and the US. Meanwhile Raja has got a job and has less time to work on this so it was stalled. Les contacted Raja and Raja agrees it is important to finish the measurements and the paper, and will endeavor to do so. Anjum is currently working on another paper. Once he finished finishes that, If Raja didn’t find time to complete the results, he will try to see if he can take over and run the code hishim-self.
- Anjum's contract has been extended for 1 year
- Saqib's contract expired last month, he is back in Pakistan.
- The 2015 ICFA/SCIC report is completed. It is available at http://www.slac.stanford.edu/xorg/icfa/icfa-net-paper-jan15/report-jan15.docx
- Hafizi Jalil will send Les his email address. Les will ask Badrul to add it to the pinger-my email list. Done 2/5/2015, Badrul has added Hafizi and added the ability for Les to manage the list. LA copy of the list is at: Membership of pinger-my
- Johari will add Ibrahim to the PingER contacts list. Done 2/5/2015
Johari has got the OK from the conference organizing committee to hold a colocated PingER/BigData workshop on August 3rd the day before the CITA 2015 (see http://www.cita.my/ an International Conference 4th - 6th August 2015, on transforming Big Data into Knowledge. Johari will provide relevant information to Bebo. Bebo will be able to make a presentation. Also Ridzuan or Ibrahim or Renan have interest in submitting a full paper by March 2nd 2015.
...
They are now preparing a test plan (like a small benchmark) to be used on all alternatives so that we can compare the results accordingly.
Les will contact Maria Luiza Campos of UFRJ (now on a one year leave of absence in Italy) to see if we can repeat the visit of Renan with a new student. Done 2/5/2015.
Les has requested (in January) Renan to provide an estimate of how DF bloats the data. Renan/Christiane are looking at this. Renan's pointed out "Finding RDF data size in bytes is not simple because it depends on which Triple Store will be used and how each triple is physically stored. One may store triples as plain texts, other may do as compressed data in specific formats, which would be much smaller."Once we have the number of PingER triples and how much the used Triple Store needs (in bytes) to store a known number of general triples, we may estimate PingER RDF data size.". Requested an update by email to Cristiane & Renan 2/4/2015.
...
Adib has been discussing with Anjum looking at potential PingER projects. Adib has a master student. In particular they are interested in providing more flexible access to PingEr PingER data rather than the limited time windows pingtable.pl provides. This may be by providing database access rather than using flat files. Les will provide:
- Access to Pinger Data in flat files via FTP. Done 2/5/2015. Les has sent information on the archive available via anonymous ftp
- Perl scripts for: making measurements, gathering the data, and analyzing the data
- Documentation on the data formats
UM
There has been no feedback to an email to Ibrahim (2/4/2015). Last we noted was:
...
- . Done 2/5/2015: Les has sent getdata.pl (the gathering script), wrap-analyze-daily.pl (the initial analysis script that takes the raw gathered data and creates hourly data points for each metric), the link to the measurement script pinger2.pl
- Documentation on the data formats. Done see: http://www-iepm
...
...
UM
There has been no feedback to an email to Ibrahim (2/4/2015). Last we noted was:
Ibrahim has setup distributed hadoop clusters. He has 2TB of disk space. Les has provided information on getting a subset of PingER data by anonymous ftp via ftp://ftp.slac.stanford.edu/users/cottrell. It was put there last September. Information on how the data was put together is IEPM/Archiving+PingER+data+by+tar+for+retrieval+by+anonymous+ftp. There is information on formatting etc at http://www-iepm.slac.stanford.edu/pinger/tools/retrievedata.html and some on the dataflows at https://confluence.slac.stanford.edu/display/IEPM/Archiving+PingER+data+flow+at+SLAC. Renan at +by+tar+for+retrieval+by+anonymous+ftp. There is information on formatting etc at http://www-iepm.slac.stanford.edu/pinger/tools/retrievedata.html and some on the dataflows at https://confluence.slac.stanford.edu/display/IEPM/PingER+data+flow+at+SLAC. Renan at UFRJ has successfully used this data, he has also characterized the data in terms of bytes/metric per year etc.
...
Johari had no updates 2/4/2015. His top priorities are:
- Reviving the Raspbery Pi
- getting Getting the research student going on anomalous behaviour detection methods.
Johari still has to uncover the problem of the traceroute from UNIMAS. UDP has been unblocked. The MYREN host works fine and share most of the hops. Thus the problem must be in the first few hops.
...
Saqib has updated the case study and is available in Google drive as a "Shared-PingER" document for review at https://drive.google.com/folderview?id=0B-NEKleLll79ZFNmUnhiVGJ0Nmc&usp=sharing_eid (
The traceroute problem regarding maximum reachable hops ( i.e. 11 hopes ) may be since the Unix/Linux/OSX traceroute uses UDP to send the requests. The first request is sent to a particular port (33434), with a ttl to tell it how many hops to go to. The ttl starts at 1 is incremented as it tries the next hop, also the port is incremented (up to 33465). It looks like the first few UDP ports are enabled and then they are blocked. The Windows traceroute uses ICMP to send the probes so does not see the problem..
NUST
Nobody on the meeting from NUST 2/4/2015.
We are We are unable to resolve the name of the hosts: pinger.uet.edu.pk, pinger.isra.edu.pk, web.hepgrid.uerj.br
pinger.uob.edu.pk appears to be partially working according to http://www-iepm.slac.stanford.edu/monitoring/checkdata/, however it appears to be unreachable at regular intervals:
...
Bebo arranged a meeting with the Colombia RENATA NREN folks and the minister of IT to discuss the use of PingER in Colombia. There is a web page at: Colombia. Les has sent an email asking them to install pinger2.pl at at least one site in Columbia.
Next meeting
Next meeting: Wednesday Mar 4th 2015 8:00pm Pacific Standard Time, Thursday Mar 5th 2015 9:00am Pakistan time, Thursday Mar 5th 2015 noon Malaysian time, Thursday Mar 5th, 2015 02:00am Rio Standard Time.
...
- Quantitative analysis on PingER data
- They want to know how PingER has grown, since 1998 until today and how it might be in the next years. By doing this, we may focus on more suitable technologies that deal with scenarios that have a similar profile with PingER.
- Two students are working on this.
- They want to know how PingER has grown, since 1998 until today and how it might be in the next years. By doing this, we may focus on more suitable technologies that deal with scenarios that have a similar profile with PingER.
- Approaches to handle PingER current data
- Conventional approach – Utilization of Cassandra as back-end database to provide easy crossing of parameters to get PingER data.
- One student is working on this.
- Distributed and parallel approach – Utilization of a data warehouse on top of a distributed file system to provide low latency response to complex queries (like the ones we were not able to do on my previous work). Additionally, how Scientific Workflow Management Systems may help in the ETL process of transforming PingER so it can easily be stored on the data warehouse.
- Renan is working on this.
- Pure RDF approach – Good ways of modeling and natively storing RDF data.
- Maria-Luiza is working on this.
- NoSQL approaches – How other NoSQL DBMS may be adequate for PingER multidimensional data.
- Two students are evaluating existing NoSQL solutions for multidimensional scenarios (such as PingER)
- Key-Value storages for PingER data in RDF
- This is Ibrahim’s work.
- Conventional approach – Utilization of Cassandra as back-end database to provide easy crossing of parameters to get PingER data.
In the end, they want to compare all these approaches.
NUST
...
- Key-Value storages for PingER data in RDF
- This is Ibrahim’s work.
In the end, they want to compare all these approaches.
NUST
Tulip
Follow up from workshop
...