Agenda for SEECS/SLAC meeting June 8th, 2011.

Bold face topics are to be addressed in the meeting.

Future & Publicity

• Arshad is putting together a 5 minute video presenting SEEC's and its research profile/activities. I recorded something via Skype  to include in this.

Pakistani case study - Zafar, Anjum

1. 36 hosts are deployed so far and are aiming to deploy 60 hosts in total in Pakistan. As I understand it SEECS will be responsible for the hosts and their PingER app## All the PERN POP nodes are on UPS, the rest of the nodes are on university power supplies which may or may not be provided with UPS
   1. Another 10 nodes are in progress and they would be active in about a month. In about two months, 17 new nodes are expected to be functional.
2. Amber and Sadia have responded to requests from Anjum for information/case studies below. Is anything else needed? 
   1. UETAXILA (see UETTAXILA Analysis of changed Network.)
   2. PERN wanted MOS further broken down by PERN and non PERN. Anwar Amjad wants an inter-city analysis. VoIP calls are relatively bad for inter-city links within Pakistan as compared to VoIP calls made outside of Pakistan (such as USA). Anwar Amjad is most interested in Islamabad to Quetta, Islamabad to Karachi and Islamabad to Lahore links. MOS analysis has to be refined further, Sadia has written a MOS report  for HEC PERN. Report here.
   3. pingerjms.pern.edu.pk is a node deployed at Jamshoro university and it will remain UP 24 hours because it has a backup power supply as it is POP. However, another node pinger.usindh.edu.pk is in the network of unversity (Non-POP) and is behaving how a network commonly behaves in that region. Amber compared the measurements of both the nodes and analyzed to if all metrics are working and also if availability is better.See Comparison of PoP (pingerjms.pern.edu.pk) and Non-PoP (pinger.usindh.edu.pk 
3. Dr. Anjum and his team at SEECS will be presenting the Pinger performance in reference to PERN and POPs to HEC (Anwar Amjad and his team).The meeting was held on Tuesday 7th June 2011. HEC was most interested in:
   1. PoP to PoP node analysis which was for intercity network visualization. RTT from say Lahore to Faisalabad and then from Faisalabad to Lahore as well. This way it will form a mesh of links.
   2. Lahore, Faisalabad, Karachi, Multan, Islamabad, Quetta and Jamshoro PoP analysis for all metrics. These PoP nodes are at PTCL and NAYATEL links which are not the educational links of HEC. HEC is planning to replace these links in about 6 months, therefore wants to know the performance of these links.
   3. PoP Vs Non-PoP analysis and confirm from traceroutes. Jamshoro node is of keen interest to HEC. Theoretically, pingerjms.pern.edu.pk should behave better than usindh. you must have enough evidence to support your results.

Latest PERN network map

Anjum got latest PERN map. Notes from Anjum: The provided map is better than the existing one but its still not complete in information and not immediately useable for our purposes. HEC PERN topology maps have yet not been received. Dr. Anjum is reminding them but no progress yet.

Status of Pakistani PingER hosts

Responsible people:

• Muhammad Talal Hussain
• Joun Muhammad

PingER managment

• Amber is starting to email contacts with non working monitors. 
• Faisal is working on a top level page for PingER.

PingER map (see http://www-wanmon.slac.stanford.edu/wan-mon/viper/pinger-coverage-gmap.html)

• Sadia is working on making a list of all monitoring nodes that ping a given node available to Faisal. Such a list is available from pingtable.pl but takes 10 secs to get and then needs filtering by Faisal's code. Thus the user has to wait 20 secs which is too long. You can also view last weeks data on these graphs but it waits for 20secs.
• Faisal is currently working on zoom able charts for todays data and is using connectivity.pl for min_rtt , avg_rtt etc.

PingER traceroute archive site

See [[http://pinger.seecs.edu.pk/cgi-bin/traceroutearchive.cgi|http://pinger.seecs.edu.pk/cgi-bin/traceroutearchive.cgi]|http://pinger.seecs.edu.pk/cgi-bin/traceroutearchive.cgi] .. some improvements needed:

• Do not include RTTs in the diffs.
• Show all changes from today's traceroute together with the first date the change was seen.
• Enable going back many more days.
• Provide better default hosts (e/g/ PERN and SEECS)
• Extend to traceroutes from SLAC.

PingER archive site -- FYP (Ghulam and Farhan)

• Revised the schema according to Umar's recommendations.
• Change in Nodes table of archive site database schema - included all information of nodes.cf file in Nodes table.
• Information of nodes comes in this table through NODEDETAILS table at SLAC.
• Data from NODEDETAILS table is collected by a script node.pl and we changed this script to store data in Nodes table instead of nodes.cf file.
• getdata.pl script has been changed to collect data from monitoring nodes and store it in to Ping_data table of archive site database instead of files.
• Changed getdata.pl script to collect nodes data from Nodes table instead of nodes.cf file because it will not be used in new architecture. The fields where sequence number or rtt is not present, NULL is used.
• Main analysis script is analyze-hourly.pl which executes daily on archive site and does analysis. This script has also been changed to get input for analysis from ping_data table instead of files. Tested and optimized.
• Other scripts analyze-daily.pl, analyze-monthly.pl, analyze-allmonths.pl, analyze-allyears.pl will be changed to get nodes information from nodes table instead of nodes.cf file. Remember these scripts are using the same data that analyze-hourly.pl is inserting into the analysis table.
• Port to SLAC

Pinger New Schema (Zafar and Co.)

• The new schema of PingER which is close to perfsonar is designed such that it stores raw data as well as analyzed data in the same table. Previous architecture of PingER first collects the data into files or table, then analyze this data and store results in another file or table. And then these hourly results are used for calculating daily and monthly results. The mechanism is as follows:*# At archive site, as the data is downloading from monitoring sites line by line, the new script does not store this data in to file on table but it uses each line's data (Basically one line contains sequence numbers, RTTs, packet size, packets sent and recieved, time and nodes information) and calculate different metrics by using data of this line.
  1. Stores these calculated metrics into arrays or keys and moves to the next line and same procedure goes again and again for every line.
  2. When all lines are processed we calculate or aggregate results on hourly basis and store all related raw data and analyzed data for this hour in one row.
  3. Daily and monthly analysis scripts which are running separately use this data (that is calculated previously) and again store results in same table but this time no raw data is stored in respective rows which contains daily and monthly data.

Adding MOS and Alpha to pingtable.pl

• Analysis scripts to add Mean Opinion Score and Alpha, some things need to be correctly configured. It has been deployed at [http://pinger.seecs.edu.pk/cgi-bin/pingtable.pl|http://pinger.seecs.edu.pk/cgi-bin/pingtable.pl] for testing.
• Alpha and MOS to be implemented at SLAC site. Sadia will be doing this with the help of Zafar

TULIP

• PlanetLab landmarks are working. the problem was in timeouts.
• TULIP is still slow. The tiering into 0 and 1 done by reflex.cgi reduced times by a factor of 2.
• Enabling/disabling of hosts appears to be working. However, the disabling of hosts has not been turned back on in th daily cronjob. Les will look at again this weekend, and see whether to turn back on.
• Reflex.cgi is ready for production use. Bilal should convert to using it. Reading documentation would bring him in good shape.
• Bilal would be working on adding clients.

CBG TULIP Integration -- FYP (Bilal)

• TULIP setup on maggie2 server and CBG is running on PERN machine.
• CBG is modified to talk to TULIP. TULIP is modified for integration.
• Details of integration are [here|https://confluence.slac.stanford.edu/display/IEPM/CBG-TULIP+Integration].* TULIP code was not running due to the machine sitting behind a proxy and unavailability of a live IP. Bilal used his home system to configure this and it worked. Replicated this to the maggie2 server. This has NAT routing and proxy policy issues. NUST HQ is dubious about external TCP traffic and thus blocks this traffic. NUST HQ is being contacted to resolve or allow traffic from reflector @ SLAC.* Latest update: Bilal is using the PERN machine (same machine being used for PerfSONAR deployment) which is on a Micronet connection for deploying CBG (Matlab code). The problem he is now facing has to do with the physical link connecting the PERN machine. A network engineer found that 50% of the packets are lost over the link and those that are not have a higher RTT (?~50ms). Micronet was contacted and they have advised to check the physical link.
• It is working .. CBG results aren't impressive. See test/development deployment at [http://203.99.52.38/cgi-bin/tulip-viz.cgi?target=www.seecs.edu.pk|http://203.99.52.38/cgi-bin/tulip-viz.cgi?target=www.seecs.edu.pk] and [http://203.99.52.38/cgi-bin/tulip-viz.cgi?target=slac.stanford.edu|http://203.99.52.38/cgi-bin/tulip-viz.cgi?target=slac.stanford.edu]
  * Other issues include: doesn't work on MSIE (Faisal), very slow (see above) and CBG is proving to be painfully inaccurate so far.
• Sadia is waiting for the MatLab license.

Best Line Approach CBG-TULIP (Zafar and Bilal):

First thing is the need of landmark sets and their RTT that can be fetched from a DB (They will be designing this DB).

Designing Database:

• need rules for defining sets . (landmark will have sets with all landmarks or will have selected ones  i.e. within a region or max RTT based )
• update frequency( daily/weekly)

Schema: one Table having RTT with pair of LandMark in each row.

Procedure: Pick landmark fromTULIP db. For each landmark

• Pick another landmark 
• Get RTT from first to second landmark
• Get RTT from second to first landmark
• Take average of both landmarks
• Apply set defining rule. 
• Insert (land1,land2,RTT) in DB table if not present.
• Update RTT if land1 and land2 are present in DB table

IPv6 activities at NUST SEECS, Pakistan

PingER2 works with IPv6, could start taking data and see what breaks. Here is the list of activities we have been doing at the moment:

• Training and Awareness: We have conducted a workshop and a seminar on IPv6. Another Seminar is planned in the coming weeks.
• Collaborations: We have created a tunnel with cybernet (local ISP) for global IPv6 connectivity. We have been allocated public IPv6 addresses by PSEB for R & D usage.
• Implementation: We are currently working for participation in world IPv6 day. Some of the services that we are currently working on are Webserver, DNS and local connectivity of IPv6 for SEECS users. We are short of dedicated hardware for this part and for this our proposal is under review. We hope to have full range of hardware covered once it is approved.
• Research: We have a team of 3-4 people working on Transition Mechanisms. Open source implementation of these mechanisms.Mobility Extensions. LISP for IPv6. Hands on implementation on NS2, GNS3.

PerfSONAR (Pakistan)

• PerfSONAR at SEECS: Problems were fixed. NTP servers were causing considerable clock delay. Added close-by Stratum 1 NTP servers to solve the problem. Nodes were updated to PerfSONAR version 3.2 (Fedora distro). Nodes however are offline since they were disrupting normal traffic. We are waiting for 10 Mbps dedicated connection to switch the nodes back on. We have a 1 Mbps link for PerfSONAR (on temporary purposes). NUST is purchasing a 2 Mbps dedicated link from WorldCall. No progress yet .. routing issues are showing live IPs as inaccessible.
• PERN will deploy perfSONAR at HEC/Quetta. Someone is working on this. The university is close by HEC/Quetta. Hope in 4 weeks to have PingER monitoring node in 4 universities in the Quetta region.
•  Dr. Anjum is trying to get some live IPs for deploying PerfSONAR, however, no progress yet.

PerfSONAR (USA)

1. Zafar is working on making SNMP-MA remote-connection-capable. It will provide the ability to read RRDs over the network instead of having the RRDs on the same machine as the RRD client. Code is working and testing is complete. Yee wants me to work on a few changes from architectural perspective.
2. Faisal is fixing bugs with the new perfSonar mashup that yee put up.
3. Sadia is installing perfSONAR server. (Not yet as she just got the machine.)

Possible projects

• See [https://confluence.slac.stanford.edu/display/IEPM/Future+Projects]. Zafar will talk to the students about these projects.
• Extend the NODEDETAILS data base to allow entry support for whether the host is currenty pingable. 
• Extend Checkdata to provide emails automatically, see [https://confluence.slac.stanford.edu/display/IEPM/Extend+checkdata+to+make+it+more+useful]. Many of the ideas in the script node-contacts.pl are a step in this direction.
• Improve the PingER2 installation procedures to make it more robust. This might be something for the person(s) in Pakistan who are responsible for installing PingER2 at the Pakistani monitoring sites. They probably have found where the failures occurs. Also look at the FAQ, and ping_data.pl which has been improved to assist in debugging, could it be further improved (e.g. provide access to the httpd.conf file so one can see if it properly configured)? There are 2 students working on the PingER archive. *Is this something they could work on?*
•  [Fix PingER archiving/analysis package to be IPv6 conformant|IEPM:Make PingER IPV6 compliant]. Will build a proposal for an IPv6 testbed. They will try various transition techniques. A proposal has been prepared and that has been submitted to PTA. Adnan is a co PI. It is being evaluated today.  A small testbed has been established in SEECS and the plan to shift some of the network to IPv6. Bilal is part of 3 students involved with PingER and they will be involved with IPv6. They are porting the PingER archive site site to using a database. They have redeveloped the archive site using Umar's documentation. They have set up a small test archive site. They have gathering, archiving, analysis. They will design a new database. They will also try a port of PingER to IPv6. 
• Look at RRD event detection based on thresholds and how to extend, maybe adding plateau algorithm. Umar's algorithm did  not work in a predictable manner. 
• Provide near realtime plots of current pinger data using getdata_all.pl/wget. It will work as a CGI script with a form to select the host, the ping size, and the time frame to plot. It will use wget or getdata_all.pl to get the relevant data and possibly RRD/smokeping to display the data. Adnan thinks one of the students working on the archive site may take this on

Future meeting time - Les

The next meeting is Wednesday, June 15th, 2011 (8 pm) for people in US and Thursday, June 16th, 2011 (8 am) for people in Pakistan.