Time & date
Wednesday July 15th 2015 10:00pm Pacific Standard Time, Thursday July 16th 2015 10:00am Pakistan time, Thursday July 16th 2015 1 pm Malaysian time, Thursday July 16th, 2015 02:00am Rio Standard Time.
Coordinates of team members:
...
? Individual emails sent
Actual attendees:
The meeting failed to transpire, Les did not make the call (his bad)!
Administration
- Membership of pinger-my in https://groups.google.com
The topic for this meeting will be plans to submit papers to NETAPPS15 (http://netapps2015.internetworks.my/v2/)
The full paper submission is July 31st, 2015 (already extended once and now a firm deadline). Papers are a max of 6 pages including figs and refs. The call for papers is at http://netapps2015.internetworks.my/v2/index.php/call-for-papers. There is information on the submission at http://netapps2015.internetworks.my/v2/index.php/paper-submission. It will be in Putrajaya 1-3 December 2015 and hosted by UUM (Adib is an organizer).
If there were sufficient papers concerning PingER than maybe there would be a separate session for PingER.
There is interest and work in progress for SLAC on "Worldwide Internet performance measurements using lightweight measurement platforms". This will be based on PingER on a Raspberry Pi, see ePinger Project at SLAC. I doubt Thiago nor I can get the travel funds. Can Johari or somebody else present?
Saqib/Anjum can you put something together on a Malaysian Case Study?
Maybe Renan, Maria, Thiago, Cristiane may put something together on Big Data and PingER including data expansion
Ibrahim /Fatima/Adib anything on PingER and Big Data/Clouds/Hadoop,
Anjum How are the measurements, analysis and paper on GeoLocation coming along?
He has been looking at the alpha (directivity) behavior, there was an exponential behavior, but it was unclear how to take advantage of it. Anjum cut N. America into regions to facilitate improving the accuracy of the alpha prediction. He believes this will improve things.
- Anjum has not started on this yet, he hopes to get to it when he returns from Canada
Johari has got the OK from the conference organizing committee to hold a colocated PingER/BigData workshop on August 3rd the day before the CITA 2015 (see http://www.cita.my/ an International Conference 4th - 6th August 2015, on transforming Big Data into Knowledge. Johari will provide relevant information to Bebo. Bebo will be able to make a presentation. Les has sent Bebo some relevant slide decks. Johari has an abstract from Bebo. There are PingER related papers submitted from UM
UFRJ
Maria Luiza has requested Raphaela and Christiane to give an update on what they've been working on in the last weeks, and maybe talk with Thiago on how they've been using their local cloud of 4 nodes with Cloudera and a data cube version of the ontology.
Cristiane reports (7/1/2015): "I am trying to automatize the triplification of PingER data on Kettle. For now, part of the transformation is made on Kettle and another is made by a Java code. Although this solution works for a data sample, is important to have the entire process on Kettle because it facilitates to understand, modify and control the triplification process."
UUM
PingER UUM is online again, problem is solved already, a new IP is assigned to UUM PingER server (http://103.5.183.4/ <http://103.5.183.4/> ). I am still testing the connectivity and stability of the new connection. Les updated the address in the meta database at SLAC, However thehost is not currently pingable
Fatima has installed hadoop on all three machines. One will stand as the host machine from which the remote machines will be controlled. She is currently trying out some MapReduce examples and Hive installation.
They have managed to get a copy of PingER data for Fatima research from UM friends, Thanks to Dr. Anjum and Mr. Ibrahim for their help and support.
Adib is involved in NETAPPS 2015. It is a forum for scientists, researchers, students, and practitioners from all over the world to present their latest research results, ideas, and developments in the area of Future Internet and discuss advancement of next generation networks. More details about NETAPPS2015 can be found in the following link: http://internetworks.my/netapps2015/v2/index.php. Included in the advisory committee are: Bebo, Anjum, Les. the conference is in December in KL. There was a discussion of how to engage PingER. There is a track on Internet protocols and service. This was agreed to be a good place to present some PingER papers on Internet measurements and a Malaysian Case Study. Saqib started a case study. It is at: https://drive.google.com/folderview?id=0B-NEKleLll79ZFNmUnhiVGJ0Nmc&usp=sharing_eid. It is incomplete and needs updating. Anjum will find a new PhD student to look at this.
NETAPPS2015 submission deadline is extended to 31/July (firm deadline), ample time to submit papers. So far, no paper has been received from PingER group. if we have enough papers, we can have one dedicated session for PingER papers and Internet performance.
UM
Badrul reports 6/28/2015: "pinger.fsktm.um.edu.my 103.18.2.152 (unable to gather data since June 26th 2015) currently has a problem with the ISP. We just changed to a new ISP last Friday and we are working on solving the problem. Also www.um.edu.my is not pingable. Anjum reports that it is old and needs replacing."
Ibrahim had downloaded PingER in Zip files format, however, when he stored them in the Hadoop distributed file system (HDFS) and try to process them, the file got corrupted, so he had to extract the file, but one file zip has more than 10000 zip files with small size. So he is trying to create a mapreduce job which can accept zip format, that will save lot of his time. Currently mapreduce can only read from files like .txt, and any doc file format or database. He will have meeting with Dr. Anjum on 11 of june asking for advice and seeking of how we can work on this together Update?
Renan reports: "I had a similar experience. HDFS works better with bigger files rather than many small files. What I did was to create a Map-Reduce job to reduce all those thousands of small files into only 17 big files, each of them containing all data for a given year [1998-2014].I didn't use Hadoop MapReduce for this, though. I used a different dataflow distributed engine that also implements map and reduce operators. I am working on providing Thiago these 17 big files. Once he gets the data, he can share them with you and explain how the data on each file are stored".
UNIMAS
Johari has a Raspberry Pi 2. It has double the RAM and a better processor. It will go to the Data Center in June. Did this happen?
Johari still has to uncover the problem of the traceroute from UNIMAS. UDP has been unblocked. The MYREN host works fine and share most of the hops. Thus the problem must be in the first few hops.
...
Saqib points out a number of Malaysian routes are IPv6 which could have problems for traceroute. Saqib is checking
MYREN
pingersonar-um.myren.net.my is down since may 31st.
Fizi had some problems with tracepath in https://perfsonar.myren.net.my/toolkit/gui/reverse_traceroute.cgi?function=tracepath. He has fixed it by changing the initial MTU.
NUST
Pink Background indicates host was bad last month, strike through says it is fixed, yellow is an new bad host.
Current status of Pakistani Hosts 7/1/2015
|
Is it time to start paring down the list of PingER monitor hosts in Pakistan, starting with those that have been down for a while and despite your efforts they are not cooperating. One might also look at the coverage by region in Pakistan and try and keep good coverage for all regions.
PingER at SLAC
Thiago has modified ping_data_plot.pl and frequency.pl to provide much improved GUI for the plots. This has been very useful for plotting the differences between pinger.slac.stanford.edu and pinger-raspberry.slac.stanford.edu. See ePinger Project at SLAC which we are putting together. the data is available at http://www-wanmon.slac.stanford.edu/cgi-wrap/pingtable.pl?file=average_rtt&by=by-node&size=100&tick=daily&year=2015&month=06&from=EDU.SLAC.STANFORD.RASPBERRY&to=WORLD&ex=none&only=all&dataset=hep&percentage=any
The Raspberry Pi at SLAC has been running smoothly since June 11th. We have power cycled it a few times to ensure it works across power outages. It is also accessible from ssh.
We have a 4 host cloud with~16Gbytes each, plus access to 220Gbytes each, Thiago is working with SLAC system people to set it up. Thiago is also working with Renan to configure things to use Hadoop and Impala.
Bebo is interested in providing PingER data of measurements to www web servers to the Web observatory (see http://webscience.org/web-observatory/). To assist in this I have added a search term to pingtable.pl to select just remote hosts with www in the name. There are 365 such hosts. The URL appears as:http://www-wanmon.slac.stanford.edu/cgi-wrap/pingtable.pl?file=average_rtt&by=by-node&size=100&tick=monthly&from=EDU.SLAC.STANFORD.N3&to=WORLD&ex=none&only=www&dataset=hep&percentage=any where only=www does the selection.
Working on the following hosts to be able to gather data
| Host | State | last seen | Status |
|---|---|---|---|
| web.hepgrid.uerj.edu | emails 12/2/2014, 12/8/2014, 2/26/2015, 4/30/2015, 6/1/2015 | Oct 23, 2014 | traceroute.pl works but no response from ping_data.pl |
| pinger.stanford.edu | email 3/14/2015 | Feb 18, 2015 | Works |
| pinger.unesp.br | email 11/28/2014, 5/22/2015, 6/1/2015. | Nov 3, 2014 | Host is pingable from SLAC. |
| pinger.fsktm.um.edu.my | email May 29th, June 6th, host in need of maintenance, Anjum may be able to assist. | May 24th | Host not pingable |
| pingersonar-um.myren.net.my | May 31st | Host not pingable |
Bebo arranged a meeting with the Colombia RENATA NREN folks and the minister of IT to discuss the use of PingER in Colombia. There is a web page at: Colombia. Les has sent an email asking them to install pinger2.pl at at least one site in Columbia. Sent a reminder email 2/27/2015. Bebo will send a gentle reminder to the RENATA people of Columbia to see whether they continue to be interested and need a meeting.
Next meeting
Next meeting: Wednesday July 15th 2015 9:00pm Pacific Standard Time, Thursday July 16th 2015 9:00am Pakistan time, Thursday July 16th 2015 noon Malaysian time, Thursday Thursday July 16th, , 2015 02:00am Rio Standard Time.
...
- Counting PingER Measurements: https://www.dropbox.com/s/35itp7v6yasy3rb/Counting%20PingER%20Measurements%20_EnglishVersion.docx?dl=0
- PingER LOD Triples: https://www.dropbox.com/s/4oj5jqupwbujja5/PingERLOD%20Triples%20_EnglishVersion.docx?dl=0
Christiane's report is at: Size Inflation of PingER Data for use in PingER LOD
...