Agenda for SEECS/SLAC meeting June

...

22, 2011.

Bold face topics are to be addressed in the meeting.

Upcoming

Zafar Gilani is leaving for Pakistan on ?July 1st, 2011.

Future & Publicity

• Faisal and Les are putting together a video on the motion charts. It is at the editing stage. 
  
• Faisal is looking at the next generation of motion charts with maps etc.

...

Dr. Anjum and his team at SEECS presented the Pinger performance in reference to PERN and POPs to HEC (Anwar Amjad and his team).The meeting was held on Tuesday 7th June 2011. HEC was most interested in:Now they want

• Write a report of PoP to PoP analysis based on the presentation showed in last meeting. Remove the outliers for future work. Add unreachability for June.
• Also compare the results of SLAC to PERN and NIIT to PERN. See if they appear different. Look at pingtable.pl to confirm the asymmetry between AIOU and LCWU. 
• Write a report of this presentation. Remove the outliers for future work. Add unreachability for June.
• Keep checking the Keep checking the traceroute for pingerjms so as to figure out why there are extra hops for pingerjms.pern.edu.pk.

...

Responsible people:

• Muhammad Talal Hussain
• Joun Muhammad
  

...

• Revised the schema according to Umar's recommendations.
• Change in Nodes table of archive site database schema - included all information of nodes.cf file in Nodes table.
• Information of nodes comes in this table through NODEDETAILS table at SLAC.
• Data from NODEDETAILS table is collected by a script node.pl and we changed this script to store data in Nodes table instead of nodes.cf file.
• getdata.pl script has been changed to collect data from monitoring nodes and store it in to Ping_data table of archive site database instead of files.
• Changed getdata.pl script to collect nodes data from Nodes table instead of nodes.cf file because it will not be used in new architecture. The fields where sequence number or rtt is not present, NULL is used.
• Main analysis script is analyze-hourly.pl which executes daily on archive site and does analysis. This script has also been changed to get input for analysis from ping_data table instead of files. Tested and optimized.
• Other scripts analyze-daily.pl, analyze-monthly.pl, analyze-allmonths.pl, analyze-allyears.pl will be changed to get nodes information from nodes table instead of nodes.cf file. Remember these scripts are using the same data that analyze-hourly.pl is inserting into the analysis table.
• Port to SLAC
  

Pinger New Schema (Zafar and Co.)

• New script for both data collection and analysis has been completed and tested.
• Testing of time complexity for a certain number of files and comparing the results for flat-files and new schema, is in progress.
• Testing results will be available in a couple of days.
• Port to SLAC
  

Pinger New Schema (Zafar and Co.)

• The new schema of PingER which is close to perfsonar is designed such that it stores raw data as well as analyzed data in the same table. Previous architecture of PingER first collects the data into files or table, then analyze this data and store results in another file or table. And then these hourly results are used for calculating daily and monthly results. The mechanism is as follows:*# At archive site, as the data is downloading from monitoring sites line by line, the new script does not store this data in to file on table but it uses each line's data (Basically one line contains sequence numbers, RTTs, packet size, packets sent and recieved, time and nodes information) and calculate different metrics by using data of this line.*# Stores these calculated metrics into arrays or keys and moves to the next line and same procedure goes again and again for every line.*# When all lines are processed we calculate or aggregate results on hourly basis and store all related raw data and analyzed data for this hour in one row.*# Daily and monthly analysis scripts which are running separately use this data (that is calculated previously) and again store results in same table but this time no raw data is stored in respective rows which contains daily and monthly data.
  

...

• PlanetLab landmarks are working. the The problem was in timeouts.
• TULIP is still slow. The tiering into 0 and 1 done by reflex.cgi reduced times by a factor of 2.
• Enabling/disabling of hosts appears to be working. However, the disabling of hosts has not been turned back on in th daily cronjob. Les will look at again this weekend, and see whether to turn back on.
• Reflex.cgi is ready for production use. Bilal should convert to using it. Reading documentation would bring him in good shape.
• Bilal would be working on adding clients.
  

...

• TULIP setup on maggie2 server and CBG is running on PERN machine.
• CBG is modified to talk to TULIP. TULIP is modified for integration.
• Details of integration are [here|https://confluence.slac.stanford.edu/display/IEPM/CBG-TULIP+Integration].* TULIP code was not running due to the machine sitting behind a proxy and unavailability of a live IP. Bilal used his home system to configure this and it worked. Replicated this to the maggie2 server. This has NAT routing and proxy policy issues. NUST HQ is dubious about external TCP traffic and thus blocks this traffic. NUST HQ is being contacted to resolve or allow traffic from reflector @ SLAC.* Latest update: Bilal is using the PERN machine (same machine being used for PerfSONAR deployment) which is on a Micronet connection for deploying CBG (Matlab code). The problem he is now facing has to do with the physical link connecting the PERN machine. A network engineer found that 50% of the packets are lost over the link and those that are not have a higher RTT (?~50ms). Micronet was contacted and they have advised to check the physical link.
• It is working .. CBG results aren't impressive. See test/development deployment at [[http://203.99.52.38/cgi-bin/tulip-viz.cgi?target=www.seecs.edu.pk|http://203.99.52.38/cgi-bin/tulip-viz.cgi?target=www.seecs.edu.pk]|http://203.99.52.38/cgi-bin/tulip-viz.cgi?target=www.seecs.edu.pk] and [[http://203.99.52.38/cgi-bin/tulip-viz.cgi?target=slac.stanford.edu|http://203.99.52.38/cgi-bin/tulip-viz.cgi?target=slac.stanford.edu]|http://203.99.52.38/cgi-bin/tulip-viz.cgi?target=slac.stanford.edu]
  * Other issues include: doesn't work on MSIE (Faisal), very slow (see above) and CBG is proving to be painfully inaccurate so far.
• Sadia is waiting for the MatLab license.
  

Best Line Approach CBG-TULIP (Zafar and Bilal):

...

• Pick another landmark 
• Get RTT from first to second landmark
• Get RTT from second to first landmark
• Take average of both landmarks
• Apply set defining rule. 
• Insert (land1,land2,RTT) in DB table if not present.
• Update RTT if land1 and land2 are present in DB table
  

Made a database named bestline with a landmarks table in it. TULIP Code is reformed such that it collects data from reflector.cgi and sites.xml and fills results in landmarks table. The data of this table is updated every time TULIP code is executed for finding a target.

Next Task is to enhance bestline.m so as to read data from table and use it for calculating geographical locations.Sadia do we have the MatLab license?

IPv6 activities at NUST SEECS, Pakistan

...