Agenda for SEECS/SLAC Meeting July 27, 2011.

• Bold face topics are to be addressed in the meeting

Upcoming

Future & Publicity

• An interview of Amber and Sadia has been published in SLAC Today:Pakistani Interns Ping SLAC . The SLAC Today article was well received in Pakistan and was published in The News : http://www.thenews.com.pk/TodaysPrintDetail.aspx?ID=59567&Cat=6&dt=7/26/2011 and Associated Press Pakistan (APP) story in the Pakistan Observer: http://pakobserver.net/detailnews.asp?id=105314
• Amber has put together an email list of PingER contacts to send the video to. We need to use it to announce motion metrics once they work again. However motion scripts is failing, we await help from Umar
• Public Data Explorer next Gen Motioncharts. Faisal wrote a PHP script to turn the PingER data into the right format. Bugs were found in the PingER data from prm.pl (duplicates). They have been fixed. Allyearly has all been re-run. Cannot see how to visualize PingER data. Need Faisal.
• Les modified prm.pl to make it case insensitive to the metric Alpha and MOS.
• Color for regions has to be added in explore data.

Pakistani case study - Amber, Anjum

1. 7 monitoring nodes have been added in the month of July and host count has reached 47 in Pakistan.
2. Has Dr. Anjum presented the case study to HEC? Do we have to d0 anything else for this case study?
3. Also has pinger map updated or no progress. 

Latest PERN network map

Anjum got latest PERN map. Notes from Anjum: The provided map is better than the existing one but its still not complete in information and not immediately useable for our purposes. HEC PERN topology maps have yet not been received.

Status of Pakistani PingER hosts

As updated on 7/27/2011.

Responsible people:

• Muhammad Talal Hussain
• Joun Muhammad

PingER managment

• wanmon.slac.stanford.edu is running out of memory and killing processes and then recovering. I am also looking in wanmon's httpd logs to correlate with the failures to get information from pingtable.pl web service. it could be due to the www.google.com/bot.html hitting wanmon.slac.stanford.edu//cgi-wrap/pingtable.pl several times per minute. We have installed robots.txt file to reduce such hits by scanners.  We are watching for re-occurences
• Sadia created a file pcm.pl which generates pcm.html. It is in production.

PingER traceroute archive site

• Sadia is working on extending to traceroutes from SLAC to the Beacon sites. Progress
• Another feature added to the traceroute at SEECS, so now it detects the loop.
• Once it works fine, she will extend it to SLAC.

PingER archive site - FYP (Ghulam, Farhan, Zafar)

• The script gets the data from monitoring sites, stores it in arrays, analyzes it, stores it in keys and the final result after analysis is placed in the table.
• Parallel code runs for collecting data and for analyzing it. Time improves because of parallel scripting.
• Only one table stores data for hourly, daily and monthly.
• it has been completed on analyze daily and monthly. currently it is taking time equal to flat file, but slows down when table is increased. they are looking for suggestions to 
• *They are currently looking into creating a view. And have compiled a report which is available at: https://confluence.slac.stanford.edu/display/IEPM/Implementation+of+Relational+archive+site+for+PingER
  They will be able to complete it by this weekend.
• A next step is to port to SLAC. This will be done by Sadia, once she is done with pcm.pl. Les is looking at the SEECS getdata.pl.

Adding MOS and Alpha to pingtable.pl

• Analysis scripts to add Mean Opinion Score and Alpha, some things need to be correctly configured. It has been deployed at http://pinger.seecs.edu.pk/cgi-bin/pingtable.pl for testing.
• Alpha and MOS to be implemented at SLAC site. Sadia will be doing this with the help of Zafar. Currently Ghulam and Farhan are working on synchronizing the SLAC and SEECS scripts.

TULIP

• TULIP (reflector.cgi) is faster since Zafar added some more parallelism. There are a couple of errors being reported which Zafar should look at.
• A tainting bug in reflex.cgi to do with regions such as Africa has been identified by Bilal and fixed by Les. Bilal is now using it. Update

CBG TULIP Integration -- FYP (Bilal)

• TULIP setup on maggie2 server and CBG is running on PERN machine.
• CBG is modified to talk to TULIP. TULIP is modified for integration.
• CBG TULIP integration is almost done. user request will be forwarded to the server(clients java code not server), and then it goes back to CBG code and displays the result in CBG browser.
• Bestline requires much larger data and the data remains same for one week, so to speed up tulip using bestline we are trying to save the data separately (not picking the data runtime). Once this is done, it will speed up the TULIP.
• Sadia is still waiting for the MatLab license. Have you talked to Justus in the last day or two?

Best Line Approach CBG-TULIP (Zafar and Bilal)

First thing is the need of landmark sets and their RTT that can be fetched from a DB (They will be designing this DB).

Currently working on bestline approach. With bestline TULIP has 70% efficiency. Bestline will run before tiering and then reflex.cgi will be used.

Next Task is to enhance bestline.m so as to read data from table and use it for calculating geographical locations.

PerfSONAR (Pakistan)

• PerfSONAR at SEECS: Problems were fixed. NTP servers were causing considerable clock delay. Added close-by Stratum 1 NTP servers to solve the problem. Nodes were updated to PerfSONAR version 3.2 (Fedora distro). Nodes however are offline since they were disrupting normal traffic. We are waiting for 10 Mbps dedicated connection to switch the nodes back on. We have a 1 Mbps link for PerfSONAR (on temporary purposes). NUST is purchasing a 2 Mbps dedicated link from WorldCall. No progress yet .. routing issues are showing live IPs as inaccessible.
• PERN will deploy perfSONAR at HEC/Quetta. Someone is working on this. The university is close by HEC/Quetta. Hope in 4 weeks to have PingER monitoring node in 4 universities in the Quetta region.
•  Dr. Anjum is trying to get some live IPs for deploying PerfSONAR, however, no progress yet.

PerfSONAR (USA)

1. Zafar: Yee will meet Jason in person (at Joint Techs in Alaska in July) to tell him about SNMP-MA extensions. He thinks emailing is not a good idea. :P
2. Les needs to ask Yee if he has met Jason and the progress on SNMP-MA extensions.
3. Zafar: Currently working on extending FTMA (which is a new PerfSONAR service and currently under developement) to add in support for MySQL.
4. Faisal is fixing bugs with the new perfSonar mashup that Yee put up.

Possible projects

• See [https://confluence.slac.stanford.edu/display/IEPM/Future+Projects].
• Extend the NODEDETAILS data base to allow entry support for whether the host is currenty pingable. 
• Extend Checkdata to provide emails automatically, see [https://confluence.slac.stanford.edu/display/IEPM/Extend+checkdata+to+make+it+more+useful]. Many of the ideas in the script node-contacts.pl are a step in this direction.
• Improve the PingER2 installation procedures to make it more robust. This might be something for the person(s) in Pakistan who are responsible for installing PingER2 at the Pakistani monitoring sites. They probably have found where the failures occurs. Also look at the FAQ, and ping_data.pl which has been improved to assist in debugging, could it be further improved (e.g. provide access to the httpd.conf file so one can see if it properly configured)? There are 2 students working on the PingER archive. Is this something they could work on?
•  [Fix PingER archiving/analysis package to be IPv6 conformant|IEPM:Make PingER IPV6 compliant]. Will build a proposal for an IPv6 testbed. They will try various transition techniques. A proposal has been prepared and that has been submitted to PTA. Adnan is a co PI. It is being evaluated today.  A small testbed has been established in SEECS and the plan to shift some of the network to IPv6. Bilal is part of 3 students involved with PingER and they will be involved with IPv6. They are porting the PingER archive site site to using a database. They have redeveloped the archive site using Umar's documentation. They have set up a small test archive site. They have gathering, archiving, analysis. They will design a new database. They will also try a port of PingER to IPv6. 
• Look at RRD event detection based on thresholds and how to extend, maybe adding plateau algorithm. Umar's algorithm did  not work in a predictable manner. 
• Provide near realtime plots of current pinger data using getdata_all.pl/wget. It will work as a CGI script with a form to select the host, the ping size, and the time frame to plot. It will use wget or getdata_all.pl to get the relevant data and possibly RRD/smokeping to display the data. 

Future meeting time - Les

1. Next meeting will be on Wednesday 10th August at 8pm in US and Thursday 11th August at 8am in Pakistan.