Minutes for SLAC SEECS Meeting February 21st, 2012

General

Amber has completed a script to automatically check the status of the nodes which are not gathering data. She is now using the page to notify admins and make minor improvements based on experience.

I have modified traceroute.pl to also provide tracepath. The new version is 5.6. if you are updating any monitors then this is the version to install. It is available at http://www.slac.stanford.edu/comp/net/traceroute/traceroute.pl

Ghulam please take your CS101 Cybersecurity at SLAC course ASAP, see http://www2.slac.stanford.edu/computing/security/education/CS101/description.htm. Let me know as soon as you complete it. Completing this course was requested Jan 20, 2012. If he does not successfully complete the course within 30 days (of Jan 20th), the account may be disabled and access to SLAC information resources may be discontinued.

IPV6 - Anjum and Ghulam (this has been de-prioritized until new database PingER is working)

IPV6 machine is working fine. Ghulam installed pinger2 on it and tried to collect data. But it was unable to resolve the IPV6 address. Seems like we need to do some changes in the whole architecture to make it workable with IPV6. Les looked at pinger2.pl, it verifies the address is IPv4 4 octets. He made a suggestion to use valid_ip.pl to verify both ipv4 and ipv6 addresses. Ghulam are there other parts of pinger2.pl that need modifiying. It already can access ping6. In addition one will need a copy of pinger.xml with IPv6 hosts and their addresses.

A possible project would be to make traceroute.pl work on a dual stack IPv6 host (say to traceroute to ipv6.google.com). Will need to look at gethostbyname etc. Is there any interest?

Another is to make pingtable.pl and getdata.pl IPV6 capable, again this could use valid_ip.pl. Since Ghulam is working on these it would be good for him to add this.

Fahad may be interested as a project next term.

pinger2.pl

Joun has installed the latest version  pinger2.pl that is not supposed to lose <BeaconsList> at 3 sites (cae.seecs.edu.pk, maggie2.seecs.edu.pk, aup.seecs.edu.pk). We will have a better idea of whether it works at the end ofb the month when the sites get updated.

Fahad and I have modified pinger2.pl so if a specific named temp file exists it is deleted and the last up date time of pinger.xml is set to zero so on the next execution of pinger2.pl it will update the pinger.xml file with the latest beacons from SLAC.   In addition we have modified ping_data.pl  to enable it to set create the temp file.

HEC Report - Anjum, Amber and Imdad

The report has been sent to Anjum. Its also placed in Case studies at https://confluence.slac.stanford.edu/display/IEPM/PERN+Six+Monthly+Report+%28June+2011-+November+2011%29. Dr Anjum met with HEC last week so we should get an update.

Imdadullah will be writing an executive summary of the report which needs to be submitted before the end of this month. Now the report goes to other universities as well as to the head of HEC. We now need to be very careful in writing the report.

Pakistani Hosts

  1. There is a discrepancy between the PERN monitors reported by Joun as not working and those SLAC is able to gather data from (reported in checkdata, here). It was reported a couple of weeks ago. Kashif and Joun are looking at it. For example we are unable to gather data from pinger.pern.edu.pk, there is something strange with the wget not working (see here). This maybe tied to some improvements to security installed on Dec 14th. Les confirmed there are still problems affecting 4 PERN POP nodes and 12 other Pakistani monitors. See Hosts pingable from some regions but not others for more details.
    1. Currently pinger.pern.edu.pk is not pingable so we are unable to test further. Progress
    2. All nodes deployed at PERN PoPs are being upgraded to enhance security. Anjum had discussed with Umar as to what security features should be implemented on these nodes. 
  2. In addition some PERN POP monitoring hosts (pingerisl-fjwu.pern.edu.pk, pingerisl-qau.pern.edu.pk, nuisb.seecs.edu.pk, nukhimain.seecs.edu.pk and pinger.pern.edu.pk ) are only pingable from Pakistan and Jordan. This may or may not be related. This needs to be resolved.  Using reflector.pl to ping nukhimain.seecs.edu.pk and also www.cern.ch, the number of landmarks able to ping nukhiman was 26, while for cern it was 106. It appears only landmarks in Pakistan, Algeria, India, Brazil, and Russia can ping nukhimain. Kashif and Joun are looking at. Progress
  3. Kashif reports we need a system for air university because they have a shortage of systems. Anjum
    The status of hosts as of 20th Jan, 8th Feb, 21st Feb 2012 is available here .

Responsible person: Joun Muhammad

HEC is sending out letters to the contact persons (who are non-cooperative) after which the nodes will be more reliable. 2-3 weeks will show much more stable nodes. Any progress?

Joun is looking at archiving the reports where we can get at them if we need to mine them.  Ghulam will be helping him in this. They resolved the issue and Ghulam will ask Joun to send the update report today.

The IPs which start with 111.68.96.xxx are routed in such a way that the data can go out but it cannot come back to Pakistan. They will change the IP of all POP nodes. This arrangement was done for NCP. Only 0.5 MB of that link is provided to other universities. Any node that is having the IP as 111.68.96.xxx will be given a new IP.

There were 12 nodes with this problem which will be solved in the next week. SEECS will give them the IP addresses, and then HEC will change the IP of these nodes.

FSBD and MTN POP have high unreachability values. which is not acceptable. They are looking into it. Backhaul network is currently leased from PTCL however in 3-4 months they will replace it with their own network. There would be no commercial traffic on it. As a result it is expected that RTT and losses will improve drastically. So next 6 months are important for observing the network perfomnce.

PingER Archive Site - Ghulam 

Ghulam: 

  1. Build database based on new schema(Send it to Sadia as well so same database can be built at SLAC)
  2. Modify getdata.pl(fine if its without parallel loops or threads as long a sit takes less than 5-6hours
  3. Run and test the getdata.pl
  4. Test it with queries as if one gives from pingtable.pl html page and measure the performance

Ghulam has rebuilt the database.

Sadia and Ghulam are working on the getdata_new.pl. Sadia has had to spend a lot of time on cleaning up the format of the code. Ghulam Has it been tested? Current status?

Current Schema : see here.

Ghulam is working on modifying getadata.pl to work without threads.

Future concerns:(Will be considered once  the performance of above monthly aggregated data is observed)

  1. How to store raw data for one year
  2. How should it be sharded
  3. For how long data should be in database

Sadia sent Ghulam the query string which worked for Sadia, he tried it and found it working as well. He will continue further.

Sadia :Adding max RTT and Alpha to pingtable.pl and the analyze scripts 

  • We need the alpha for identifying strange Pakistani routes. This will be done before we move to the new schema, it cannot wait anymore. Sadia has modified analyze hourly and analyze daily for Max RTT, MOS & alpha. Now she has to run for all days back to 1998. She is running the jobs in batch. This is in progress
  • Ghulam there was some problem in pingtable.pl alpha value. For some links alpha was having value of 200 . As we know alpha can have maximum of value 2. So there must be something wrong in calculation. Progress

TULIP - Bilal

Bilal will be delivering three tasks in this week:

  1. Trying stress testing with reflector instead of reflex. Results are available at Target Data for reflector tier all. The results also have  a comparison of reflex and reflector error in terms of distance. Repeat this for Europe. 
  2. We need to look into why reflex is not working as good as reflector. 
  3. By the next week Bilal will be submitting report on Australia using both reflector and reflex. He will also try reflector for Europe.
  4. Bilal will be sending the tulip draft paper by the end of this month.

There is interest from Northwester University in using Tulip. The version at SLAC still needs the MatLab license so we recommend they use the SEECS version, e.g. for http://203.99.52.38/cgi-bin/tulip-viz.cgi?target=132.206.6.88. It does not look like the SEECS version uses reflex.pl. Bilal had a chat with the person from North Western University explaining him how Tulip and CBG works.

What is the story on MatLab Sadia.

Also there is a problem with a key needed for Google maps. Sadia

Looking at the SEECS version it does not appear to be using the tiering properly  - Bilal.

PerfSONAR (Pakistan)
  • Bilal and Ghulam will have a meeting with Zafar to know about PerfSonar and to maintain it in future. Update?

Possible projects

  • There can be a paper kind of talking on Pinger if we could just find the right conference. MCN, ICC and Globecomm do provide network monitoring topics. We can talk of GEO-Location experiences. For example within Pakistan it works fine, however as we go within regions or continents this gets worse. We can publish some stats on that for example. We are yet not ready for Tulip paper.
  • See [https://confluence.slac.stanford.edu/display/IEPM/Future+Projects].
  • Extend the NODEDETAILS data base to allow entry support for whether the host is currenty pingable. 
  • Extend Checkdata to provide emails automatically, see [https://confluence.slac.stanford.edu/display/IEPM/Extend+checkdata+to+make+it+more+useful]. Many of the ideas in the script node-contacts.pl are a step in this direction.
  • Improve the PingER2 installation procedures to make it more robust. This might be something for the person(s) in Pakistan who are responsible for installing PingER2 at the Pakistani monitoring sites. They probably have found where the failures occurs. Also look at the FAQ, and ping_data.pl which has been improved to assist in debugging, could it be further improved (e.g. provide access to the httpd.conf file so one can see if it properly configured)? There are 2 students working on the PingER archive. Is this something they could work on?
  •  [Fix PingER archiving/analysis package to be IPv6 conformant|IEPM:Make PingER IPV6 compliant]. Will build a proposal for an IPv6 testbed. They will try various transition techniques. A proposal has been prepared and that has been submitted to PTA. Adnan is a co PI. It is being evaluated today.  A small testbed has been established in SEECS and the plan to shift some of the network to IPv6. Bilal is part of 3 students involved with PingER and they will be involved with IPv6. They are porting the PingER archive site site to using a database. They have redeveloped the archive site using Umar's documentation. They have set up a small test archive site. They have gathering, archiving, analysis. They will design a new database. They will also try a port of PingER to IPv6. 
  • Look at RRD event detection based on thresholds and how to extend, maybe adding plateau algorithm. Umar's algorithm did  not work in a predictable manner. 
  • Provide near realtime plots of current pinger data using getdata_all.pl/wget. It will work as a CGI script with a form to select the host, the ping size, and the time frame to plot. It will use wget or getdata_all.pl to get the relevant data and possibly RRD/smokeping to display the data. 
  • See above for potential IPv6 projects.

Future meeting time - Les

  1. Next meeting on Tuesday 28nd February, 2012 at 8:00 pm in US and Wednesday 29rd February, 2012 at 9:00am in Pakistan.
  • No labels