...
Wednesday February March 5, 2014 8:00pm (or 9:00pm ) Pacific Standard Time, Thursday March 6th 2014 9:00am (or 10:00am ) Pakistan time, Thursday March 6th, 2014 noon (or 1:00pm ) Malaysian time, Thursday March 6thy6th, 2014 023:00am (or 3:00pm) Rio Standard Time.
Attendees
Invitees:
Anjum+, Hassaan Khaliq, Kashif+, Raja+, Johari+, Nara, Abdullah, Badrul-, Ridzuan+, Ibrahim, Hanan, Saqib+, Adib-, Les+, Renan, Bebo+
+ Confirmed attendance
- Responded but Unable to attend:
Actual attendees:
...
Anjum, Kashif, Raja, Ridzuan, Ibrahim, Saqib, Bebo, Les
Hassaan was not notified about the meeting since he is not on the pinger-my email list. Les will send email to Badrul to get him added.
Administration
Anjum is
Administration
Anjum is at UM. He does have access to the Internet from the Lab but its a shared place and he cannot attend meetings while in the Lab. For meetings, either he shall attend it from his home, or from the common room of the building.
Johari proposes that UTM- UUM are interested to host a PingER workshop at UUM in 2014, maybe associated with another conference.
Adib suggested a summer school that is aimed at graduate students and researchers from different disciplines where everyone needs to be present for team presentations based on case studies/assignments given to each team. Each team will be mentored by one professional researcher (from PingER Current Member) to guide and advise. The Summer School activities will include: Keynote talk, Presentations on current projects related to Pinger that open up challenges for students to solve and address, and case studies (assignments based on the presentations) for each team of students.
- Johari talked to Adib who is interested in hosting the workshop at UUM. They believe that with Anjum in Malaysia, the main cost is for Les. Johari shared the information from the previous 2 workshops and it is believed it is good value for money. Given this, it appears the funding required is achievable. There may also be a possibility for a presentation by Les to a more public audience at the ITU Research Center, as well as the possibility of getting sponsors and requiring a registration fee for private sector people. Adib will put together a proposal and take it to the UUM Vice Chancellor.
Renan
Renan realized that testing on Virtuoso is going to take longer than he thought because of the big number of triples. He did not have enough time to continue his tests.
Renan completed writing the paper on Linked Open data access to PingER data. Bebo 7 less have reviewed and commented. It was submitted to the Stanford conference March 1st, 2014.
Renan did not not see Vinicius (the admin of the UFRJ PingER monitor). We can try to contact him via email again (maybe in the end of February?). If he doesn´t reply, Renan can try to talk to him in person again.
It would be good if somebody else was able to work with Renan to move PingER LOD forward. However we think the performance issue has to be resolved first.
UM
UM appears to be experiencing very high packet losses (>4%). We need help in understanding these. It does not look like simple congestion since the jitter and RTT is staying low. Saqib needs assistance from Badrul as part of his case study of Malaysia. Badrul writes: "UM is now currently changing the new policy regarding the bandwidth and connection. They are upgrading the switch and router to a new router - mostly every weekend they shut down the network. therefore we are now having some difficulty." Saqib has identified what appear to be losses starting at a JARING router in KL. Badrul will run some mtr traceorute/pings to various sites.
Badrul plans for his undergraduate student (Abdulrahim Haroun Ali) to have a draft paper on anomalies in PingER measurements to share for review by the end of semester (January). Update?
Ridzuan is still in the configuration process for communication between the Hadoop nodes. Need some troubleshooting at the moment.
Ibrahim Abaker is planning to work on topic initially entitled " leveraging pingER big data with a modified pingtable for event-correlation and clustering", The purpose of this project is to make Pinger more efficient and more scalable to retrieve and analysis pingER raw data. Consequently it may improve pinger data store. He is planning to use key-value store model with high expandability and shorter query response time. The benefit of such project will help in the design of real time analysis and distributed pinger data across multiple servers for analysis. Les has sent him a pointer to documentation on the data flow at SLAC and also listings of pingtable.pl and associated script. He is currently writing a proposal, which will contain a problem statement, objectives, scope and the approach to tackle the problem. He asked: how much PingER data is captured and stored so far (~ 1TByte)? Is there a published paper (see http://www-iepm.slac.stanford.edu/paperwork/ieee/ieee.pdf)?
UNIMAS
No progress 2/19/2014.
The Raspberry Pi is at the data centre, but due to miscommunication, the public ip which was previously assigned to the rasp pi unit has been assigned to another server. So now Johari has to wait for the ip to be reassigned back to rasp pi unit and give new ip to the other server. They suggested a new ip for the rasp pi unit but Johari wanted to have the old one so that we do not have to update the record at slac for the archival process, etc
The tool to enable synchronizing Malaysian monitors is completed. It provides the ability to add, edit and delete nodes. Raja has tested and after a mod was made to replace a private IP address it is working well. Badrul and Saqib need to try it out.
The traceroute server at http://pinger2.unimas.my/cgi-bin/traceroute.pl has the same problem as before. They know (sort of) the problem but haven't got the chance to rectify it (mapping NAT address, needs to be added). There is no progress 12/4/2013, 1/8/2013, 1.22.2014, 2/5/2014. Now that the historical traceroutes are working for UM (see below) there is an extra incentive to get the reverse traceroute working at UTM and UNIMAS
Johari has a research student who finalized a proposal in order to officially apply for his masters. He will start in February. He is currently working on threshold/anomaly detection, and will extend to correlating performance over multiple routes. He will share the proposal with Les and others in the next 2 weeks to get reviews.
UTM
No progress 2/19/2014
Saqib received an email regarding the successful selection of research assistant-ship at UTM. However, he is waiting for an official offer letter from research management center.
Saqib is working with the IT department in UTM to solve the problems of delays in traceroute. They have a new ISP. Now the pinger server is working with single static IP (161.139.68.188) both for intranet and internet. Previous internet IP 161.139.146.158 is removed. Again given the new historical traceroute facility (see below), it is important to get this working.
- No load balancing is involved as UTM is working with single ISP.
- Still the problem of the delay in traceroute is not solved. It appears to be due to the DNS lookup delay.
Saqib has started a case study of Malaysian hosts seen from Malaysian Monitors. The case study is identifying the need for the Malaysian monitors to collect traceroutes to their target hosts daily. SLAC has a script to facilitate this, for routes seen from SLAC, that could be adapted for other sites. The latest copy of the word file was shared with Badrul and Johari. No update 2/5/2014.
UUM
Dec 4, 2013: The director of computer centre has agreed to support this collaboration and already appointed one staff to be with us during the configuration. Next step, Adib needs to discuss with Prof. Suhaidi to dedicate one PC OR get support from Badrul grant to buy one PC for this purpose or have a bake sale and buy a Raspberry Pi. Adib will do his best to setup PingER monitoring host at UUM before the end of this year. Adib was not on the meeting. He provided an update by email: "Still waiting to get PC. But they promise to provide one very soon. it is totally beyond my control." Regarding the meeting, Adib will attend the upcoming meetings once UUM pinger host is up. No update 2/5/2014.
NUST
At the Connect Asia Pacific Summit in Bangkok in Januaryt and seeing the project "Mapping the pan Asia Pacific information Superhighway and closing gaps in infrastructure connectivity" Shahryar found that very much related to the work in the PingER project. So Shahryar sent email to a UN agency for a possible collaboration with them on PingER project. He has heard nothing so he will write a detailed proposal and then should contact them again. No update 2/5/2014.
Hassaan reports that they have established contact with people in Bahawalpur. The contact person forwarded the request to higher authorities for approval of PingER deployment.
The contact at Sahiwal did the installation of PingER node by himself but data is not being collected, so we are in contact to resolve the issue. Kashif believes he missed some Linux updates. Today is holiday here and he will do the trouble shooting on 6th Feb.
We were gathering data from Pingerisl-air (this is continuing to work), and Quest (stopped working Feb 13)
We are still unable to gather data from airuniversity, buitms, cae, duhs, pinger.pern.edu.pk, pingerisl-qau, sau.seecs.edu.pk , uaf.
Monitors Status
Airuniversity New installation will be install in a week
Buitms Problem in hardware, they are trying to fix it. System Replacement Required
Cae Problem with IP, they are trying to resolve it
Duhs They are purchasing new servers in two week time
pinger.pern.edu.pk Fixed Feb 19.
Pingerisl-qau Contact person is still on leave. He is now busy in other pending works.
sau.seecs.edu.pk for all of January. Contact person is not attending
Uaf Problem in system, visit approval for Faisalabad University in progress. Approved for 12th Feb
Kashif reports (2/19/2014): "We are in a process of shifting us along with equipment to another lab from a week. Due to this transition process and new installations, Joun could not contact further to the contact persons about the down PingER Nodes. This shifting process will complete on coming Friday and after that i will be able to update you about the task on NUST part."
TULIP - Raja
...
- . However, they have two issues, one is the budget (they need to run this workshop on cost recovery basis); second is their commitment to MYIGF which is hosted by UUM (btw 12-13 august). There was no update 3/5/2014.
Renan
Renan realized that testing on Virtuoso is going to take longer than he thought because of the big number of triples. He did not have enough time to continue his tests.
Renan completed writing the paper on Linked Open data access to PingER data. Bebo 7 less have reviewed and commented. It was submitted to the Stanford conference March 1st, 2014. The deadline for the submission was extended from March 1st to March 15. The notification is May 1st, the final submission is May 9th, conference is May 27-31. Poster submissions are due April 4th, we might submit a poster just in case. The acceptance rate is very low. Information on the conference is at http://www.scienceengineering.org/ase/conference/2014/bigdata/sanjose/website/paper-submission/.
Les will try and set up a Skype meeting with Renan and Bebo to go over strategy and also to discuss Ibrahim's work (see below).
Les will send another email to Vinicius and Renan to see if we can get the UFRJ host running agai (the web server is not working).
It would be good if somebody else was able to work with Renan to move PingER LOD forward. However we think the performance issue has to be resolved first.
UM
UM appears to be experiencing very high packet losses (>4%). We need help in understanding these. It does not look like simple congestion since the jitter and RTT is staying low. Saqib needs assistance from Badrul as part of his case study of Malaysia. Badrul writes: "UM is now currently changing the new policy regarding the bandwidth and connection. They are upgrading the switch and router to a new router - mostly every weekend they shut down the network. therefore we are now having some difficulty." Saqib has identified what appear to be losses starting at a JARING router in KL. Badrul will run some mtr traceorute/pings to various sites. No update 3/5/2014.
Badrul plans for his undergraduate student (Abdulrahim Haroun Ali) to have a draft paper on anomalies in PingER measurements to share for review by the end of semester (January). No update 3/5/2014.
Ridzuan successfully setup hadoop environment for the servers. He is now looking into how best to input the pinger data to the environment and type of analytics that can be done. Ridzuan is working through some issues due to lack of IPv4 addresses and the need to use IPv6.
Ibrahim Abaker is planning to work on a topic initially entitled " leveraging pingER big data with a modified pingtable for event-correlation and clustering", The purpose of this project is to make Pinger more efficient and more scalable to retrieve and analysis pingER raw data. Consequently it may improve pinger data store. He is planning to use key-value store model with high expandability and shorter query response time. The benefit of such project will help in the design of real time analysis and distributed pinger data across multiple servers for analysis. Les has sent him a pointer to documentation on the data flow at SLAC and also listings of pingtable.pl and associated script. He is currently writing a proposal, which will contain a problem statement, objectives, scope and the approach to tackle the problem. He asked: how much PingER data is captured and stored so far (~ 1TByte)? Is there a published paper (see http://www-iepm.slac.stanford.edu/paperwork/ieee/ieee.pdf)? Bebo & Les will set up a Skype meeting with Renan to get an update and see about setting up direct contact between Renan and Ibrahim. Ibrahim provided a copy of his proposal (see https://confluence.slac.stanford.edu/download/attachments/17162/leveraging%20pingER%20big%20data%20with%20a%20modified%20pingtable%20for%20event-correlation%20and%20clustering.docx).
UNIMAS
No progress 2/19/2014. Johari was not on the call.
The Raspberry Pi is at the data centre, but due to miscommunication, the public ip which was previously assigned to the rasp pi unit has been assigned to another server. So now Johari has to wait for the ip to be reassigned back to rasp pi unit and give new ip to the other server. They suggested a new ip for the rasp pi unit but Johari wanted to have the old one so that we do not have to update the record at slac for the archival process, etc
The tool to enable synchronizing Malaysian monitors is completed. It provides the ability to add, edit and delete nodes. Raja has tested and after a mod was made to replace a private IP address it is working well. Badrul and Saqib need to try it out.
The traceroute server at http://pinger2.unimas.my/cgi-bin/traceroute.pl has the same problem as before. They know (sort of) the problem but haven't got the chance to rectify it (mapping NAT address, needs to be added). There is no progress 12/4/2013, 1/8/2013, 1.22.2014, 2/5/2014. Now that the historical traceroutes are working for UM (see below) there is an extra incentive to get the reverse traceroute working at UTM and UNIMAS
Johari has a research student who finalized a proposal in order to officially apply for his masters. He will start in February. He is currently working on threshold/anomaly detection, and will extend to correlating performance over multiple routes. He will share the proposal with Les and others in the next 2 weeks to get reviews.
UTM
No progress 2/19/2014
Saqib received an email regarding the successful selection of research assistant-ship at UTM. However, he is waiting for an official offer letter from research management center.
Saqib is working with the IT department in UTM to solve the problems of delays in traceroute. They have a new ISP. Now the pinger server is working with single static IP (161.139.68.188) both for intranet and internet. Previous internet IP 161.139.146.158 is removed. Again given the new historical traceroute facility (see below), it is important to get this working.
- No load balancing is involved as UTM is working with single ISP.
- Still the problem of the delay in traceroute is not solved. It appears to be due to the DNS lookup delay.
Saqib has started a case study of Malaysian hosts seen from Malaysian Monitors. The case study is identifying the need for the Malaysian monitors to collect traceroutes to their target hosts daily. SLAC has a script to facilitate this, for routes seen from SLAC, that could be adapted for other sites. The latest copy of the word file was shared with Badrul and Johari. No update 2/5/2014.
UUM
Adib was unable to attend the meeting. He provided an update by email: "The PingER monitor will be up soon"
NUST
The PingER monitor at SEECS/NUST (maggie1) is being rebuilt on a different computer following a crash. There are 2 approaches thta can be tried in parallel:
- Rebuild from scratch. in the initial case Umar took 2-3 weeks. he wrote documentation, so this will be a test of the documentation. This could be valuable should Malaysia or another site decide to set up an archive. kashif and Joun are ready to start on this.
- Dump and restore the database. This is much easier. However it requires the database password. Les will contact Umar to see if he can provide the password. Umar canot logon to maggie1 since it is blocked by the firewall.
The BeaconLists at NUST and SLAC are out of sync. Raja will compare the two pinger.xml files. It turns out NUST was not using the HostList for the non Beacons and so had an expanded BeaconList which was updated from SEECS. Raja has explained this to NUST and believe it will be easy to synchronize.
Hassaan reports that they have established contact with people in Bahawalpur. The contact person forwarded the request to higher authorities for approval of PingER deployment. Same status 3/5/2014
The installation of PingER at Sahiwal is complete. It just needs the Beacon List.
We were gathering data from Pingerisl-air (this is continuing to work), and Quest (stopped working Feb 13)
We are still unable to gather data from airuniversity, buitms, cae, duhs, pingerisl-qau, sau.seecs.edu.pk , uaf.
Monitors Status
Airuniversity New installation will be install in a week
Buitms Problem in hardware, they are trying to fix it. System Replacement Required
Cae Problem with IP, they are trying to resolve it
Duhs They are purchasing new servers in two week time
pinger.pern.edu.pk Fixed Feb 19.
Pingerisl-qau Contact person is still on leave. He is now busy in other pending works.
Quest.seecs.edu.pk down since February 20th.
sau.seecs.edu.pk for all of January. Contact person is not attending
Uaf Problem in system, visit approval for Faisalabad University in progress. Approved for 12th Feb
Anjum pointed out that there are working PingER monitors that SLAC appears unable to gather data from. An example is the Comsats host. Les takes this to mean pinger.comsats.edu.pk and will investigate.
At the Connect Asia Pacific Summit in Bangkok in January and seeing the project "Mapping the pan Asia Pacific information Superhighway and closing gaps in infrastructure connectivity" Shahryar found that very much related to the work in the PingER project. So Shahryar sent email to a UN agency for a possible collaboration with them on PingER project. He has heard nothing so he will write a detailed proposal and then should contact them again. No update 2/5/2014, 3/5/2014.
TULIP - Raja
Please try out http://www-wanmon.slac.stanford.edu/cgi-wrap/reflector.cgi?function=vtrace It is a visual traceroute. It uses TULIP to locate the routers. It is in beta test and needs testing by friendly users. We are looking to add a link to it from the perfSONAR home page.
Raja has added 56 more perfSONAR landmarks, most of them are in North America. Currently we have 298 active landmarks (from total of 918).
The Visual traceroute and TULIP have been modified to support IPv6. Also Raja has improved the laundering.
Raja is working on a paper on TULIP implementation and on the Visual traceroute that uses TULIP.
Raja is adding a web page to select a landmark to request it to ping a target.
- Tool for pinging from PlanetLab landmarks http://www-wanmon.slac.stanford.edu/cgi-wrap/reflector.cgi?function=plping
- TULIP landmark list with Ping and Traceroute URLs http://www-wanmon.slac.stanford.edu/cgi-wrap/reflector.cgi?function=
...
- landmarks
- TULIP landmark map with Ping and Traceroute URLs http://www.slac.stanford.edu/comp/net/wan-mon/viper/tulipmap.html
We have sent a reminder email to Yahoo giving them an update on TULIP and how it might be extended to meet their needs. They have responded, they appear to be interested. We are trying to interest them in providing landmarks. No update 2/18/2013.Raja has added 56 more perfSONAR landmarks, most of them are in North America. Currently we have 298 active landmarks (from total of 918)providing landmarks. No update 2/18/2013.
PingER at SLAC
We put together a slide presentation to ICFA in Geneva, to be given by Harvey Newman. It is at: http://www.slac.stanford.edu/xorg/icfa/icfa-net-paper-jan14/ICFASCICPresentation_MonitoringGroupSlides20140205.pptx
Raja has also re-written the historical traceroute facility that takes nightly traceroutes and allows one to retrieve and compare them. He then extended it to use the reverse traceroute facility to provide histories for NUST, CERN, UM as well as SLAC. When the UNIMAS and UTM traceroutes work, we can include them. See for example http://www-wanmon.slac.stanford.edu/cgi-wrap/traceroutearchive.cgi
Three is an empty pinger/xml file at ICTP. Les & Raja are studying. The Pinger hosts at FIU in Florida is now restored. Les is contacting the Jordanian PingER monitor contact to restore it.
Old Items
Linked Open Data
...
Anjum suggested Saqib, Badrul and Johari put together a paper on user experiences with using the Internet in Malaysia as seen from Malaysian universities. In particular round trip time, losses, jitter, reliability, routing/peering, in particular anomalies, and the impact on VoIP, throughput etc. It would be good to engage someone from MYREN.
Potential projects
Future meeting - Les
Next meeting Wednesday March 5th 19th 2014 8:00pm Pacific Standard Time, Thursday March 6th 20th 2014 9:00am Pakistan time, Thursday March 6th20th, 2014 noon Malaysian time, Thursday March 6th20th, 2014 02:00am Rio Standard Time.
...