...
Hassaan Khaliq?, Kashif, Raja, Samad Riaz (SEECS); Johari-, Nara, Adnan Khan (UNIMAS); Abdullah, Badrul, Anjum+, Ridzuan, Ibrahim+ (UM); Hanan, Saqib- (UTM); Adib+, Fatima+ (UUM); Fizi Jalil (MYREN); Thiago, Les+, Bebo+ (SLAC)
+ Confirmed attendance
...
? Individual emails sent
Actual attendees:
Anjum, Fatima, Ibrahim, Bebo, Les, Adib was unable to attend due to an overrunning NetApps meeting
Administration
- Membership of pinger-my in https://groups.google.com
We are working with Dr Zaidi the new head of SEECS to investigate resources need to continue support of PingER in Pakistan. We communicated first week in August. No word since. Sent a reminder 8/1/2015. We had a rump meeting of Anjum, Hassaan and Les to discuss the way forward. Hassaan is waiting to hear from HEC about the comments on the proposal.
- NETAPPS2015: Adib reports "We have received some good number of submissions and the reviewing process have started already. we will have a committee meeting tomorrow at the same time of PINGER group meeting. I will update you afterward."
Four papers were submitted related to PingER August 25th
Anjum submitted a paper to NETAPPS2015 "Adaptive Geolocation of Internet Hosts". There is no word on acceptance of paper. The evaluation was complete by 23rd September. I think the results should be out soon. It has been accepted.
Les, Thiago, Johari, Bebo and Topher White submitted a paper on "Worldwide Internet Performance Measurements Using Lightweight Measurement Platforms". We have heard nothing on acceptance.
Saqib submitted a paper "PingER Malaysia-Internet Performance Measuring Project: A Case Study". Saqib has been notified that it has been accepted. Now trying for some funding to attend the workshop.
Thiago and the Brazilian team submitted a paper on "Applying Data Warehousing and Big Data Techniques to Analyze Internet Performance'. Adib has kindly come up with a way in which the authors are unlikely to be able to attend, so somebody else will present. This has been accepted. Thiago is working on responding to the reviewer comments and is updating the paper.
Bebo has potentially 9 students from Amity University near New Delhi in India interested in coming to SLAC for 3 months starting Feb until May. All expenses paid by them. They can start working remotely on project immediately. We are looking at what they can be involved in: PingER (e.g. data warehouse project follow on with Brazil, Maris is excited about this), Android port of PingER MA with Topher, Geolocation with Anjum (Anjum is interested), others on big data, monitoring, high performance throughput etc. However to come to SLAC they will need a US visa and they are still undergraduates so SLAC cannot sponsor them for a visa. Bebo is contacting the students to see whether any of them already have a BS or equivalent (or higher). Or whether any have a US visa or knows how to get one without SLAC sponsorship. A possibility is that they collaborate remotely until they get a degree and/or can get a visa..
At a previous meeting, we discussed the potential for an undergraduate from UM spending 6 months at SLAC as an intern. SLAC would be very interested, however SLAC cannot sponsor a J1 visa for anyone who does not have the equivalent of a US bachelors degree. Anjum will pass this on and pursue from the UM end. It does nor appear promising, we will drop it.
Renan identified a potential Brazilian student for next year. We have responded to Rena that we are very interested.
Need to move next meeting(s) 30 Need to move meeting 30 minutes earlier in future to accomodate Johari's schedule.
...
Anjum believes the TULIP Geolocation application can be improved significantly. At least there are few ideas that we can try. For this, either a group of undergraduate students or an active masters student is required. The resultant work can easily be the thesis of masters level. Who is interested?
Saqib at Faisalabad has an MS student interested to work on Geolocation project. He requests an initial paper on the project.
Johari will contact Anjum to learn more of the requirements. Update Johari/Adnan
Les has responded to Saqib. He also has some other students. Anjum will contact him. Potential projects/asks include: take over management of PingER monitoring in Pakistan (say 5 monitors/student; case study of how Pakistan's network performance/connectivity has improved over thea years especially as function of funding etc; geolocation with variable alpha; indoor geolocation
Johari will contact Anjum to learn more of the requirements. Update Johari/Adnan
See http://www.See http://www.slac.stanford.edu/comp/net/tulip/. Basically TULIP uses pings to a target from landmarks at known locations and converts the minimum RTTs to estimate the distances. Then uses the distances with mulitlateration to estimate the location of the target
To improve TULIP one needs the right selection of landmarks, i.e. good (working landmarks) at the right locations (not too far from the target), straddling the target, and with a a reasonable estimate of the indirectness (directivity or alpha) of the path from the landmark to the target (so we can reasonably accurately estimate the distance). One also needs a reasonable density of landmarks (e.g. number of targets/100,000sq km)
The landmarks come from PingER and perfSONAR sites. We have a reasonable density in the US, Pakistan and Europe. Currently Anjum is getting better than 20km accuracy for Pakistani targets
As the number of landmarks goes up so does the accuracy, but so does the time to make the measurements (pings).
One needs to find the optimal density
Anjum proposes to speed up the measurements using a cluster for parallelization and also proposes to improve the adaptation of alpha based region. He regards the adaptive geolocation and parallelization as MS projects.
He is also interested in geolocation in small proximity (e.g.indoors), e.g. using cell tower signals. This is a new area of research. It is possible that the port of PingER to an Android could be related to this. This is a PhD project
Anjum reports he can supervise the students on Geolocation. He will need to know when the students are ready. We can start with a joint meeting involving Les and the students. Later on, Anjum can have the meeting with students every week while Les can join if he has time.
...
Bebo said Topher would be interested in getting a student to port PingER to an Android. Possibly one of the Chinese students. Les put out a version of the PingER MA on Github at https://github.com/iepm/ in 2013 as part of coordinating with Google( which did not get very far). It has been dormant since it was put out there. Les is not a github expert or user so is unclear how complete it is or who can access it. Can somebody test and if not available let Les know what to do.
Topher is at home for 2 weeks, he will set up a shareable Github for PingER files.
Unrest in Malaysia - Anjum, Johari, Les
- Anjum wondered if PingER could detect anomalies related to the recent unrest (protests against the government and DOS attacks against universities (such as UM or UNIMAS) in Malaysia. there were several attacks from the Anonymous group between August 28-31st. Following the meeting Les put out a web page at: Malaysian unrest Aug-Sep 2015. Is there an update?
UFRJ
UUM
- Adib has managed to resolve almost all issues blocking UUM pinger from running smoothly including the problematic switch connecting Measurement Agent (including, firewall settings, DNS, etc.) . It should be stable from now on.
from running smoothly including the problematic switch connecting our lab with the computer center, firewall setting, DNS, ..etc. Therefore, we hope not to have any connectivity problem. - They plan to move UUM pinger to UUM computer center, I have talked to the person in charge and we just need to get the director permission to move on. This will help us to resolve the electricity problem as well as to some how guarantee stable connectivity.
- Looking at the log and data at SLAC it looks to be working since 29th September.
Fatima implemented her work (involving HDFS and MapReduce with Pinger Data) on 50MB to 200MB data sets, using a 2-node cluster, and then a 3-node cluster. The work is not completed, but she has obtained some good results and is putting together the presentation for her final viva. She and Ibrahim are using the same cloud/cluster at MYREN.
UM
Ibrahim had downloaded PingER in Zip files format, however, when he stored them in the Hadoop distributed file system (HDFS) and try to process them, the file got corrupted, so he had to extract the file, but one file zip has more than 10000 zip files with small size. So he is trying to create a mapreduce job which can accept zip format, that will save lot of his time. Currently mapreduce can only read from files like .txt, and any doc file format or database. He will have meeting with Dr. Anjum on 11 of june asking for advice and seeking of how we can work on this together. No update 9/2/2015. Anjum will contact Ibrahim.Renan reports: "I had a similar experience. HDFS works better with bigger files rather than many small files. What I did was to create a Map-Reduce job to reduce all those thousands of small files into only 17 big files, each of them containing all data for a given year [1998-2014].I didn't use Hadoop MapReduce for this, though. I used a different dataflow distributed engine that also implements map and reduce operators. I am working on providing Thiago these 17 big files. Once he gets the data, he can share them with you and explain how the data on each file are stored".has extracted the PingER Zip manually. He is reconstructing 11Gb of data, heas 15GBytes of data there. He is trying to use SPAR to classify the data. He alsop was looking at RDF. the next step si to use MapReduce to organize and reduce the output of dataso can vizualize it. He will be using a teh same techniques he used for looking at 1996-2006 weather data.
UNIMAS
...
Has a student interested to work on Hadoop and pinger data. Les has sent him information on the SLAC Nebula cluster Cloudera/impala/HDFS Pinger Data warehouse dedveloped by Thiago and Co.
Further he has a few more MS students searching for their research topics. We can also utilize them in our pinger project. They are from software engineering, database and network background.
5. Further, I have few more MS students searching for their research topics. We can also utilize them in our pinger project. They are from software engineering, database and network background.
MYREN
No update 8/12/2015, no Update 9/2/2015.
...