Versions Compared

Old Version 3

changes.mady.by.user Les Cottrell

Saved on

compared with

New Version Current

changes.mady.by.user Les Cottrell

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Time & date 

Wednesday  May 21st,  2014 9:00pm Pacific Standard Time, Thursday may 22nd 2014 9:00am Pakistan time, Thursday May 22nd, 2014 12:00 noon Malaysian time, Thursday May 22nd, 2014 1:00am Rio Standard Time.

Attendees

Invitees:

Anjum+, Hassaan Khaliq, Kashif+, Raja+,  Johari+, Nara, Abdullah, Badrul, Ridzuan+, Ibrahim, Hanan, Saqib+, Adib, Les+, Renan, Bebo

+ Confirmed attendance

- Responded but  Unable to attend: 

Actual attendees:

Anjum, Hasaan, Kashif, Raja, Saqib, Les

Administration

  • Adib is waiting to hear from colleague (Dr. Nordin: chair of ISTT2014), there was no update as of this meeting. The conference is 2 days so the workshop may be squeezed to one day. The conference will be held in Langkawi between 24-26 Nov.  

    Hopefully, this will help us to attract good number of participants and get sponsor/support to cover our expenditure. Other option is to conduct the workshop in UUM, we are ready to provide Local Arrangement (lab, local transportation, and food). But there is no funding provided by the school  to support the transportation/accommodation of the  speakers.

  • Anjum will try and contact Adib. Anjum is also putting together a conference on cloud computing in November. I may be possible to co-locate the PingEr workshop with this conference.

  • Les will be in Burkina Faso Nov 13-21. He flies back to London on Nov 22nd.  He can get to Malaysia mid morning November 24th. Thus the workshop should be at the end of the ISTT2014 meeting if Les is to attend.

Renan

  • Anjum suggested putting together a paper on metrics provided by PingER for Sigmetrix. The due date is in November.

Renan

Renan 'Renan 's  paper on "Linked Open Data Publication Strategies: Application in Networking Performance Measurement Data" submitted  to the Academy of Science 2014 Bigdata / SocialCom / Cybersecurity conference at Stanford May 27-31 has been accepted (last year acceptance was 8.5%) for a poster session May 29th. See http://www.scienceengineering.org/ase/conference/2014/bigdata/sanjose/website/program/.  Renan and his supervisor will be coming to Stanford for the meeting .and will present the paper

UM

Badrul is still awaiting hearing from his student (Abdulrahim Haroun Ali) on  the paper on anomalies in PingER measurements  and will update later once the paper ready. 

Ridzuan has put together a rough proposal to use Hadoop to store and make available PingER data.  There is a rough draft at: https://confluence.slac.stanford.edu/download/attachments/17162/leveraging%20pingER%20big%20data%20with%20a%20modified%20pingtable%20for%20event-correlation%20and%20clustering.docx. Ridzuan requested a copy of the PingER data.  Les & Raja put together a repository of the minimum_rtt by hour data since Jan 1998 and made it available by FTP.There was a discussion about using hadoop and other hosts joining. Also the MYREN cloud  has public access to Malaysian academic institutions may be of interest. Johari will mail some details to Ridzuan. He is evaluating several vendors version of Hadoop 2.0 installations to decide which is the right one to adopt. Last time he successfully installed and tried Hadoop 1.0 but then it has many drawbacks such as no real-time streaming processing.  Johari sent information on the Myren Cloud. The web site  is as follows: https://cloud.myren.net.my/. It gives information about the service and how to apply, plus user guide is available from the website. As for Ridzuan, he should be eligible to register for the service, but he needs to check with  UM (University Malaya) procedures for applying for the service. 

Ibrahim Abaker  is planning to work on a topic initially entitled " leveraging pingER big data with a modified pingtable for event-correlation and clustering".  Ibrahim has a proposal, see https://confluence.slac.stanford.edu/download/attachments/17162/leveraging%20pingER%20big%20data%20with%20a%20modified%20pingtable%20for%20event-correlation%20and%20clustering.docx. He is currently spending  time reading about RDF storage and its retrieval, also he is  trying to setup Hadoop clusters for the experiment. He is in email discussion with Renan on which part to work on first. He plans to  collaborate with Renan to make the pinger store and process more efficient. Les has sent him documentation of PinGER which is very helpful. Ridzuans' work is more to do with hosting the data and stream data analysis. Ibrahim is looking more at applying MapReduce ( programming model for processing large data sets with a parallel, distributed algorithm on a cluster), reducing the storage needs and providing querying of the data.

...

Johari has a research student who finalized a proposal in order to officially apply for his masters.  He will start in February. He is currently working on threshold/anomaly detection, and will extend to correlating performance over multiple routes. He will share the proposal with Les and others April.

UTM 

Saqib is seeing some unexplainable results from the traceroute servers at UTM. Basically traceroute from UTM works fine with -I option. Otherwise it stop at hops 10 or 11 unless he uses the -f 6 option. He documented by email and Les added it to the wiki at Traceroute at UMT. The traceroute problem is under discussion with MYREN and CICT UTM.

Saqib had a detailed discussion regarding the Packet loss at UM with Myren.  They suggest that packet loss January 11-13 of >4% loss  may be due to congestion in the UM edge router (perhaps due to high traffic or traffic filtering/shaping). They suggest to  pursue this with UM IT Division to know what is hop 11. Saqib has emailed Badrul.

The Case study is almost done. Saqib has included the packet loss analysis at Um in the case study. Waiting for the reply of MYREN and CICT to include in it. Need one more week. 

No updates regarding traceroute problem at UTM. However, Saqib thinks the problem is still in CICT firewall or router as tcp traceroute command works fine from UTM.

The Malaysian Case study is completed. Saqib has included the packet loss analysis at UM in the case study.  Les has reviewed and sent suggestions. Saqib will incorporate.

Unfortunately the FRGS proposal was rejected as Saqib's Project leader does not have any relevant publications in his record. Saqib proposes to submit another in ERGS or E-Science grants. He is in discussions Saqib received feedback on the submitted FRGS 2014 proposal ("A Fundamental Active Internet Performance Monitoring Framework for Malaysian Research & Education Network") for 36 months.  He will update it and re-submit. He proposes to submit another in ERGS or E-Science grants. He is in discussions with his PhD. superviser to get the requirements and has started on the proposal. a proposal to submit elsewhere

UUM

Regarding the monitoring host in UUM, Adib has assigned one student to prepare the configuration/installation plan including how to secure our host from attack. He has a public IP address.  He needs to the DNS registration by Sunday or Monday.  He is in the last stage of working with the Computer Center. Adib requested Johari to share  the UNIMAS setting so it is easier for the student to follow. 

NUST

Kashif has copied new beacon list (modified pinger.xml for pakistani nodes) at  the HEC-PoP node at Khi (pingerkhi.pern.edu.pk). By looking at the data it seems to be allowing  pings outside Pakistan. This implies, its working fine and there is no blockage (May be for some outside Pakistan nodes but not for all).  So  the new synchronized beacon list has no problem so far.

All references for the Bahawalpur site goes to Director IT, but he is still All references for the Bahawalpur site goes to Director IT, but he is still silent and we are waiting for his reply.

Kashif reports the  installation of PingER at Sahiwal is complete. Raja will add to PingER and TULIP (site is not responding - might be a power outage 11:42AM 4/23/14 Pacific Time). Added as PingER monitor and traceroute/ping server.

The following nodes (were down for months) and have now The following nodes (were down for months) has been recovered by the efforts made by Kashif in the last two weeks and they are working fine now. These all are also updated with the new beacon list by replacing pinger.xml file.

...

.

...

  1. pinger.nca.edu.pk
  2. pinger.nwfpuet.edu.pk
  3. pinger.pern.edu.pk (HEC)
      pingerfsbd

      1. pinger2 command was working on "pinger.pern.edu.pk

    1. pingerisl-fjwu.pern.edu.pk

      1. " but due to space shortage it was not storing data. Now Kashif removed some older files and it starts working again, also the time of the server was not correct so he changed that too.

    3. pingerfsbdpingermtn.pern.edu.pk (node is ok but down again n again due to power outage)
    4. pingerisl-fjwu.pern www.upesh.edu.pk
    5. buitmspinger.seecslcwu.edu.pk is also fine. collection status at SLAC will change the color today after collecting data for 19th May arround 4pm PKST. 
    6. pingermtn.pernmaggie1.seecs.edu.pkpk 
        monitor
        1. pingermtn.
        seecs
        1. pern.edu.pk

      Kashif is working on the rest of nodes.  Among those may be the following that have not responded to gathering data in the last month are:

        1. remained DOWN for most of the time due to power outage in last week. Due to sudden breakdown, the Linux OS was also creating problem at reboot and get stuck in checkdisk type of errors and was demanding manual clicks and commands.  So, in this situation, Kashif requested to the contact person and he managed to arrange UPS for the PingER monitoring node. Hopefully it will UP now most of the time or at least till the backup time of UPS, if power outage remains for multiple consecutive hours.
      2. pinger.uob.edu.pk is working, but having multi-hour power outages each day. This usually means that when SLAC normally goes to gather the data in the early morning Pacific time the host does not respond so we are unable to gather the data.  Analyzing the outages, it it appears that a good time gather the data is at ~11:00am Pacific time, since the host usually responds at that time.
      3.  www.upesh.edu.pk
      4. buitms.seecs.edu.pk
      5. maggie1.seecs.edu.pk
      6. monitor.seecs.edu.pk
        1. Also needed the time corrected and updated pinger2.pl and pinger.xml

      Kashif is working on the rest of nodes. He plans to have 3 or 4 more up and running this coming week..  Among those, may be the following that have not responded to gathering data in the last month are:

      ...

      To assist in SEECS gathering data getdata: nodes.cf is now available via http://www-iepm.slac.stanford.edu/pinger/tools/nodes.cf.slac.stanford.edu/pinger/tools/. Also to assist in debugging some problems with gathering and checking the data the SLASC versions of checkdata.pl and checkdata_gif.pl are available at the same place.

      Raja

      Raja, Umar and Les are working on a have submitted the paper on TULIP and its application to Visual Tracerouting to be submitted at end of April.

      PingER at SLAC 

      ...

      the Vancouver conference in November. Aceptances are due in July. Raja is improving the GUI for TULIP. In particular visualizing the intersection region (showing both the minimum and maximal circles), showing landmarks, adding Google analytics, and looking at providing results from undns (a tool for guessing the location from the router name) for VTrace.

      PingER at SLAC

      Les requested an update from Yahoo about TULIP's gelocationgeolocation. They answered "We are very much interested in getting IP triangulation at internet scale, we will have internal sync-up on how we can leverage this initiative if there is rate limit and get back. Regarding opening up yahoo sites for deploying ping server requires some more time to discuss this with relevant stake holders with in yahoo." No word holders with in yahoo." No word, sent a reminder 5/19/2014.

      Les sent email to Google as follows: "I would like to bring to your attention that we have developed a geolocation tool using delay based (using RTTs from known ping server landmarks) distance estimates to triangulate the location of an IP host target. The tools is accessible at: http://www-wanmon.slac.stanford.edu/cgi-wrap/reflex.cgi. We have identified that the accuracy of the geolocation is directly related to the landmark density (e.g. # of landmarks/ million sq km). The higher the density the smaller the error and the fall off is exponential. We currently have over 1000 registered landmarks, of which at any given time ~300 are working. The tool not onlyonly finds the location of the target, it also gives an estimated error. To the best of our knowledge it is the only freely available delay based measurement geolocation tool service publicly available today. A drawback (compared to database methods such as those based on GeoMind) is the time taken to make the measurements. We have worked in this from manydirections including parallelization of the ping requests, caching, tiering to get the rough location (i.e. region of the world) then zooming in using all landmarks in the region. We are putting together a publication on this."Les contacted the  Hong Kong, Rutherford Lab (.uk), Daresbury Lab (.uk), CEFET (.br, UNESP(.br) and ACMESecurity (.br) PingER monitors to try and get them fixed. The Hong Kong contact has retired. The others have not responded. Les will probably disable the nodes.to make the measurements. We have worked on this from many directions including parallelization of the ping requests, caching, tiering to get the rough location (i.e. region of the world) then zooming in using all landmarks in the region. We are putting together a publication on this."

      Old Items

      Linked Open Data

      ...

      Anjum suggested Saqib,  Badrul and Johari put together a paper on user experiences with using the Internet in Malaysia as seen from Malaysian universities. In particular round trip time, losses, jitter, reliability, routing/peering, in particular anomalies, and the impact on VoIP, throughput etc.  It would be good to engage someone from MYREN.

      Potential projects

      See list of Projects

      Future meeting  - Les

      Next meeting Wednesday May 7th June 4th  2014 9:00pm Pacific Standard Time, Thursday May 8th June 5th 2014 9:00am Pakistan time, Thursday May 8thJune 5th, 2014 noon Malaysian time, Thursday May 8thJune 5th, 2014 01:00am Rio Standard Time.

      Coordinates of team members:

      ...