Minutes for SLAC SEECS Meeting March 6th, 2012
General
We may not be able to get the MATLAB Toolkit for Tulip. Dr. Anjum proposed that we can use Mathemetica instead of MATLAB. He also proposed that he will try to find out some MS students who might take this conversion as his MS thesis. Students have to finalize their thesis by 6th April. By the second week of April, we will get to know if we have any student doing this.
Bilal and Dr. Anjum were unavailable this week, pre-informed. Ghulam did not show up.
IPV6 - Anjum and Ghulam (this has been de-prioritized until new database PingER is working)
IPV6 machine is working fine. However, we cannot logon. Ghulam is investigating. Progress
Les has the passwords for monitor and the IPv6 host at SEECS. He has successfully logged on.
Earlier Ghulam installed pinger2 on it and tried to collect data. But it was unable to resolve the IPV6 address. Les looked at pinger2.pl, it verifies the address is IPv4 4 octets. In addition one will need a copy of pinger.xml with IPv6 hosts and their addresses. I have modified traceroute.pl to work with IPv6. The new version is 5.71. if you are updating any monitors then this is the version to install. It is available at http://www.slac.stanford.edu/comp/net/traceroute/traceroute.pl. I am unable to fully test at SLAC since we have no IPv56 host/network. It has 2 subroutines There are 2 subroutines in traceroute.pl that may be useful in making pinger2.pl support IPv6. They are valid_ip and gethostbyname6.
Next we need to make pingtable.pl and getdata.pl IPV6 capable, again this could use valid_ip.plthe 2 subroutines. Since Ghulam and Sadia are working on these it would be good for them to add this.
We were unable to log in to IPV6 machine at SEECS. Kashif sattar sent an email to Les for sharing username and ?passwords for Maggie1 and Maggie2 on 29th Feb, but we haven't yet acted upon it.
pinger2.pl
Joun has installed the latest version pinger2.pl that is not supposed to lose <BeaconsList> at 3 sites (cae.seecs.edu.pk, maggie2.seecs.edu.pk, aup.seecs.edu.pk). We will have a better idea of whether it works at the end of the month when the sites get updated.
What are the updates on these 3 hosts, Joun? Amber send joun an email and ask.
We have defined how to force updates of the <BeaconList> fro SLAC. However, it is not implementedA month later (march 7, 2012) after install the <BeaconsList> appears OK on all three, i.e. for the <LastChecked> variable showed march 1st, 2012 for aup.seecs.edu.pk and maggie2.seecs.edu.pk, and showed March 7th, 2012 for cae.seecs.edu.pk. Joun may want to extend the deployment of the new pinger2.pl.
PingER Updates
- Collection status webpage generates an email if a node does not work for more than 10 days. put it on web saying for Update the script to report bug that for first n days we do not get report.
- Connectivity.pl is fixed to display the latest data (of hours of day that have passed) results on the web.
- Harway Harvey Newman sent new measurement results of HDI for 2011. These new results have been added to both Motion charts and Pinger Data explorer. Both of them are working fine.
- Amber is working on creating smokeping plots for today's data.
HEC Report - Anjum, Amber and Imdad
The report is in Case studies at https://confluence.slac.stanford.edu/display/IEPM/PERN+Six+Monthly+Report+%28June+2011-+November+2011%29.
...
The report was supposed to be submitted last monday. A few changes were made in the conclusions and an executive summary. We wait till next meeting for updates.
Pakistani Host
Joun and Ghulam have arranged to archive the statu sof status of Pakistani hosts in http://pinger.seecs.edu.pk/daily-report. It appears future copies be saved there*. There is a link from the PingER web site (http://www-iepm.slac.stanford.edu/pinger/site.html)
The IPs which start with 111.68.96.xxx are routed in such a way that the data can go out but it cannot come back to Pakistan. They will change the IP of all POP nodes. This arrangement was done for NCP. Only 0.5 MB of that link is provided to other universities. Any node that is having the IP as 111.68.96.xxx will be given a new IP.
...
SEECS have not heard about the IP change from HEC. Dr Anjum will remind HEC about the IP changes to be made by the end of last week. We wait till nest week for updates.
PingER Archive Site - Ghulam
Current Schema : see here.
...
- How to store raw data for one year
- How should it be sharded
- For how long data should be in database
Sadia :Adding max RTT and Alpha to pingtable.pl and the analyze scripts
- We need the alpha for identifying strange Pakistani routes. This will be done before we move to the new schema, it cannot wait anymore. Sadia has modified analyze hourly and analyze daily for Max RTT, MOS & alpha. Now she She has to run the jobs in batch for all days back to 1998. She is running the jobs in batch. This is in progress. Gathering of data is complete. Next Next step in to analyze the output of data files. There does not appear to be any data in http://www-wanmon.slac.stanford.edu/cgi-wrap/pingtable.pl?file=average_rtt&by=by-node&size=100&tick=hourly&year=2012&month=03&from=EDU.SLAC.STANFORD.N3&to=WORLD&ex=none&only=all&dataset=new&percentage=any
- Ghulam there was some problem in pingtable.pl alpha value. For some links alpha was having value of 200 . As we know alpha can have maximum of value 2. So there must be something wrong in calculation. Progress
TULIP - Bilal
Bilal will be delivering three tasks in this week ending 3/7/2012:
- Trying stress testing with reflector instead of reflex. Results are available at Target Data for reflector tier all. The results also have a comparison of reflex and reflector error in terms of distance. Repeat this for Europe.
- We understood and fixed the problem with reflex missing most of the landmarks. It is much faster.
- By the next week Bilal will be submitting report on Australia using reflex. He will also try reflector for Europe. Australia report using reflex can be found at: https://confluence.slac.stanford.edu/display/IEPM/Stress+Testing+for+AustraliaLes has provided Bilal with a way to get the lat longs of more hosts in Australia
- . This required a minor change to HostSearcher.pl
- Bilal needs to add the number of landmarks in the region for the EXcel Spreadsheet. This will be useful for his paper, i.e. reporting typical accuracy as a function of landmarks in region.
- Bilal will be sending the tulip draft paper by the end of this monthFebruary.
There is interest from Northwestern University in using Tulip. The version at SLAC still needs the MatLab license so we recommend they use the SEECS version, e.g. for http://203.99.52.38/cgi-bin/tulip-viz.cgi?target=132.206.6.88. It does not look like the SEECS version uses reflex.pl. Bilal had a chat with the person from Northwestern University explaining him how Tulip and CBG works.It appears Sadia is right about SLAC needing to spend ~ $4K to get the full toolkit needed by CBG. Thus we will not implement at SLAC. Sadia will make sure TULIP art SLAC works for TULUP without CBG. Progress
Stress testing for Australia region is completed using reflex but not reflector. North America is completed with reflector and reflex. However we got to know this week that reflex was having lesser landmarks because PlanetLab landmarks weren't added. Bilal will rerun the stress testing of North America using new reflex.
Bilal is writing a report with stress testing results of all of the regions. He will be sharing it when it completes.
With the upgrading of the pinger.slac.stanford.edu host from RHEL4 to RHEL6 the mysql databases got lost. Sadia has worked with Fahad to recover the Tulip data base from the recovered sites.xml file. This is almost done.
Possible projects
- There can be a paper kind of talking on Pinger if we could just find the right conference. MCN, ICC and Globecomm do provide network monitoring topics. We can talk of GEO-Location experiences. For example within Pakistan it works fine, however as we go within regions or continents this gets worse. We can publish some stats on that for example. We are yet not ready for Tulip paper.
Wiki Markup - Extend the NODEDETAILS data base to allow entry support for whether the host is currenty pingable.
Wiki Markup - Improve the PingER2 installation procedures to make it more robust. This might be something for the person(s) in Pakistan who are responsible for installing PingER2 at the Pakistani monitoring sites. They probably have found where the failures occurs. Also look at the FAQ, and ping_data.pl which has been improved to assist in debugging, could it be further improved (e.g. provide access to the httpd.conf file so one can see if it properly configured)? There are 2 students working on the PingER archive. Is this something they could work on?
- [Fix PingER archiving/analysis package to be IPv6 conformant|IEPM:Make PingER IPV6 compliant]. Will build a proposal for an IPv6 testbed. They will try various transition techniques. A proposal has been prepared and that has been submitted to PTA. Adnan is a co PI. It is being evaluated today. A small testbed has been established in SEECS and the plan to shift some of the network to IPv6. Bilal is part of 3 students involved with PingER and they will be involved with IPv6. They are porting the PingER archive site site to using a database. They have redeveloped the archive site using Umar's documentation. They have set up a small test archive site. They have gathering, archiving, analysis. They will design a new database. They will also try a port of PingER to IPv6.
- Look at RRD event detection based on thresholds and how to extend, maybe adding plateau algorithm. Umar's algorithm did not work in a predictable manner.
- Provide near realtime plots of current pinger data using getdata_all.pl/wget. It will work as a CGI script with a form to select the host, the ping size, and the time frame to plot. It will use wget or getdata_all.pl to get the relevant data and possibly RRD/smokeping to display the data. Amber is looking at.
Future meeting time - Les
- Next meeting on Tuesday 13th March, 2012 at 8:00 pm in US and Wednesday 14th March, 2012 at 9:00am in Pakistan.