Minutes for SEECS/SLAC Meeting August 17, 2011.

• Anjum, Zafar, Bilal and Ghulam were not in the meeting.

Upcoming

SEECS has an IPV6 Live IP through cybernet. They are currently testing the compatibility issues of Pinger Node/software on it.The mechanism may start as: Take a new host and set it up as IPV6. Then monitor other hosts on IPV6 like ipv6 google etc. Once data is available then work on how to analyze it and use it. Anjum asks, If we have IPV6 and it tries to monitors IPV4, will that work or there will be some problem with that. Les thinks it will work fine. He suggests that we might first put up a separate copy on a host such that it monitors another IPV6 host  and then check how it monitors IPV4. Anjum is going to ask Joun to configure Pinger node on IPV6. Zafar if available, would be helping Joun in this case. Update?

SLAC now has an IP project with a project manager just hired. Things may start to happen soon.

Future & Publicity

• Amber has put together an email list of PingER contacts to send the video to. We need to use it to announce motion metrics once they work again. However motion script is failing, we await help from Umar. Les has made contact with Umar who is going to look at the problem.
• Public Data Explorer next Gen Motioncharts. Amber has written a Perl script, which is dual of Faisal's PHP script, to turn the PingER data into the right format. Need to format it after .
• Public Data Explorer is working now.
• Added Average RTT and Packet Loss, internet users.  Next thing is adding colors to the regions.
• Once colors are added, documentation to Pinger map script will be the next task.

Pakistani case study - Amber, Anjum

1. 3 monitoring nodes have been added in the month of August and host count has reached 50 in Pakistan.
2. HEC is shifting the intercity links on Pern2 (so far intracity links are on Pern 2 while intercity are on PTCL). HEC is expecting to have an analysis on what is the effect on min rtt and other metrics after the links are shifted from PTCL to Pern2. (POP to POP is on PTCL).
3. Amber keep a track of POP to POP links, additional POP links will be intracity. (intercity POPs and intracity POPs analysis will be carried out for HEC later on). Intercity POP nodes have yet not been added and so unable to carry out the said analysis.

Latest PERN network map

Anjum got latest PERN map. Notes from Anjum: The provided map is better than the existing one but its still not complete in information and not immediately useable for our purposes. HEC PERN topology maps have yet not been received. Should we give up on this?

Status of Pakistani PingER hosts

As updated on 8/17/2011.

PingER Management

Before Farhan, Bilal and Ghulam Nabi leave, a documentation and demonstration will have to be given to the others. We need some continuity for the next few months. They will be hired for the next three months.

PingER traceroute archive site

• Sadia is working on extending to traceroutes from SLAC to the Beacon sites. The autotrace.pl for collecting the traceroutes from SLAC to all the nodes is completed, however it is not on cronjob. Testing for errors.
• Traceroute.cgi still needs to be tested because it requires cgi wrap directory which is currently not accessible to Sadia. It now passes the laugh test of Tainting warning and strict. It will be moved to a production place and then Les will get it moved to the web cgi-wrap space. Will be adding error check to this code as next task.

PingER archive site - FYP (Ghulam, Farhan, Zafar)

• The archive site based on relational database is complete and is ready to be deployed on the server.
• Issues related to speed and performance (in pingtable.pl ) are solved by using MySql database features like Indexing and Caching queries.
• Daily, monthly as well as yearly analysis scripts are developed based on relational database and now all the flat-files based architecture is removed.
• Daily and Monthly analysis results (for DB) are prepared and their comparison graphs with flat-files are also prepared. How do they look? we cannot try this out from SLAC if the server is at SEECS because we still do not have the Live IP.
• Some comparisons between flat files and relational database were made for daily , monthly and yearly analysis. Farhan will send these results to Zafar and Dr. Les.
• Deployment of new Relational archive site on Server will be deployed on SEECS whenever the access to the server is provided to farhan. He still has not got the server with live IP.
• A next step is to port to SLAC. This will be done by Sadia, once she is done with pcm.pl. Les is looking at the SEECS getdata.pl.

Adding MOS and Alpha to pingtable.pl

• Analysis scripts to add Mean Opinion Score and Alpha, some things need to be correctly configured. It has been deployed athttp://pinger.seecs.edu.pk/cgi-bin/pingtable.pl for testing.
• Alpha and MOS to be implemented at SLAC site. Sadia will be doing this with the help of Zafar. Currently Ghulam and Farhan are working on synchronizing the SLAC and SEECS scripts.

TULIP

• TULIP (reflector.cgi) is faster since Zafar added some more parallelism. There are a couple of errors being reported which Zafar will look at. Update?

CBG TULIP Integration -- FYP (Bilal)

• Bilal has observed that if he gives more than three targets simultaneously in different tabs then SLAC only provides data for two instance others don't get data from SLAC. It may  be a connection pooling limit problem. Les has replied him in email. Update?
• TULIP setup on maggie2 server and CBG is running on PERN machine.
• CBG is modified to talk to TULIP. TULIP is modified for integration. (Email Bilal to know the update. And ask him to )
• CBG TULIP integration is almost done. user request will be forwarded to the server(clients java code not server), and then it goes back to CBG code and displays the result in CBG browser.
• Bestline requires much larger data and the data remains same for one week, so to speed up tulip using bestline we are trying to save the data separately (not picking the data runtime). Once this is done, it will speed up the TULIP.
• Matlab licence is here; Sadia, Les, Nick and Justus need to meet to discuss the installation issues. Where are we with this?

Best Line Approach CBG-TULIP (Zafar and Bilal)

First thing is the need of landmark sets and their RTT that can be fetched from a DB (They will be designing this DB).

Currently working on bestline approach. With bestline TULIP has 70% efficiency. Bestline will run before tiering and then reflex.cgi will be used.

Next Task is to enhance bestline.m so as to read data from table and use it for calculating geographical locations.

PerfSONAR (Pakistan)

• PerfSONAR at SEECS: Problems were fixed. NTP servers were causing considerable clock delay. Added close-by Stratum 1 NTP servers to solve the problem. Nodes were updated to PerfSONAR version 3.2 (Fedora distro). Nodes however are offline since they were disrupting normal traffic. We are waiting for 10 Mbps dedicated connection to switch the nodes back on. We have a 1 Mbps link for PerfSONAR (on temporary purposes). NUST is purchasing a 2 Mbps dedicated link from WorldCall. No progress yet .. routing issues are showing live IPs as inaccessible.
• PERN will deploy perfSONAR at HEC/Quetta. Someone is working on this. The university is close by HEC/Quetta. Hope in 4 weeks to have PingER monitoring node in 4 universities in the Quetta region.
•  Dr. Anjum is trying to get some live IPs for deploying PerfSONAR, however, no progress yet.

Possible projects

• The problem is that many hosts do not give high priority to pings and many block it. This results in high RTT for pings. It is a big difference for closer hosts than far hosts. Note the response time of pings vs HTTP hosts from slac to other hosts.
• There can be a paper kind of talking on Pinger if we could just find the right conference. MCN, ICC and Globecomm do provide network monitoring topics. We can talk of GEO-Location experiences. For example within Pakistan it works fine, however as we go within regions or continents this gets worse. We can publish some stats on that for example. We are yet not ready for Tulip paper.
• See [https://confluence.slac.stanford.edu/display/IEPM/Future+Projects].
• Extend the NODEDETAILS data base to allow entry support for whether the host is currenty pingable. 
• Extend Checkdata to provide emails automatically, see [https://confluence.slac.stanford.edu/display/IEPM/Extend+checkdata+to+make+it+more+useful]. Many of the ideas in the script node-contacts.pl are a step in this direction.
• Improve the PingER2 installation procedures to make it more robust. This might be something for the person(s) in Pakistan who are responsible for installing PingER2 at the Pakistani monitoring sites. They probably have found where the failures occurs. Also look at the FAQ, and ping_data.pl which has been improved to assist in debugging, could it be further improved (e.g. provide access to the httpd.conf file so one can see if it properly configured)? There are 2 students working on the PingER archive. Is this something they could work on?
•  [Fix PingER archiving/analysis package to be IPv6 conformant|IEPM:Make PingER IPV6 compliant]. Will build a proposal for an IPv6 testbed. They will try various transition techniques. A proposal has been prepared and that has been submitted to PTA. Adnan is a co PI. It is being evaluated today.  A small testbed has been established in SEECS and the plan to shift some of the network to IPv6. Bilal is part of 3 students involved with PingER and they will be involved with IPv6. They are porting the PingER archive site site to using a database. They have redeveloped the archive site using Umar's documentation. They have set up a small test archive site. They have gathering, archiving, analysis. They will design a new database. They will also try a port of PingER to IPv6. 
• Look at RRD event detection based on thresholds and how to extend, maybe adding plateau algorithm. Umar's algorithm did  not work in a predictable manner. 
• Provide near realtime plots of current pinger data using getdata_all.pl/wget. It will work as a CGI script with a form to select the host, the ping size, and the time frame to plot. It will use wget or getdata_all.pl to get the relevant data and possibly RRD/smokeping to display the data. 

Future meeting time - Les

1. Next meeting will be on Wednesday 24th August at 8:30 pm in US and Thursday 25th August at 8:30am in Pakistan.