Agenda for SEECS/SLAC meeting May 26th, 2011.

Bold face topics are to be addressed in the meeting.

Future

Pakistani case study - Zafar, Anjum

1. 27 more hosts are yet to be deployed, currently at ~ 37 will go to 60.
2. Sadia Updated the MOS reports for PERN-PERN, PERN to non-PERN, see  here. This is suffiicinet forr Anjum. Anjum is waiting tio hera from Anwar to schedule. 
3. To assist in identifying PERN nodes Sadia has updated the affinity groups and procedures and add a PERN group and a PERN-POP group. This is documented at http://www-dev.slac.stanford.edu/cgi-wrap/scriptdoc.pl?name=create-group.pl
4. pingerjms.pern.edu.pk is a node deployed at Jamshoro university and it will remain UP 24 hours because it has a backup power supply as it is POP. However, another node pinger.usindh.edu.pk is in the network of unversity (Non-POP) and is behaving how a networkcommonly behaves in that region. Compare the measurements of both the nodes and analyze. Check if all metrics are working and also check if avaialbility is better. Amber will be doing this. Done. See Comparison of PoP (pingerjms.pern.edu.pk) and Non-PoP (pinger.usindh.edu.pk). We need to verify the tracreoutes and min RTT. 
5. Has anyone at PERN been approached about giving someone from HEC the responsibility for ongoing support. Concern will be that once the nodes are deployed, we will have all the latest patches but since these nodes will be on internal network and it could be a potential security hazard once security is breached. Umar suggests that we should get this in writing from HEC that they'll be responsible for these nodes on their POPs. This is probably in the noise compared to the HEC problem. Anjum will send email to Anwar Amjad or one of his lieutenants  to ID someone to manage the nodes. No word. Managing of node is responsibility of NUST after project is closed.
6. Putting monitoring hosts on UPS. Anjum is following on this with PERN. Just the POP nodes.
7. Dr. Anjum and his team at SEECS will be presenting the Pinger performance in reference to PERN and POPs to HEC (Anwar Amjad and his team). Sadia and Amber will be helping him. Dr. Les would be providing input on what should be included in presentation that could be of interest to HEC. Few things that he would like we should present are: MOS analysis as discussed , UET Taxila analysis, more concrete findings in terms of high RTT for various nodes within Pakistan, separate analysis of PoP nodes. How did it go? Is more needed?

Latest PERN network map

Anjum got latest PERN map. Notes from Anjum: The provided map is better than the existing one but its still not complete in information and not immediately useable for our purposes. HEC PERN topology maps have yet not been received. Dr. Anjum is reminding them but no progress yet.

Status of Pakistani PingER hosts

Amber states that the SEECS team have been very responsive in sorting out these hosts.

Responsible people:

• Muhammad Talal Hussain
• Joun Muhammad

PingER

• Google spreadsheets may be used to reduce friction in information gathering process for new hosts. Done. Faisal mailed Dr. Les about it.

• Amber is starting to email contacts with non working monitors. 
• Amber is trying to make contact with the Mexican PingER monitor, the email addresses do not work. There has been no response, we will probably have to give up on this important site.
• Amber is working on connectivity.pl and improving mon-lib.pl as a useful Perl learning exercise. Connectivity.pl is now working again, we have added Excel (csv) output for the individual pairs. this is needed by Faisal. We still have to add the Alias name. Amber will do this.
• Sadia has cleaned up the groups so they are mainly regional and so static.Sadia has added deletion of affinity groups. It works for pingtable.pl. This has been updated at SEECS also. The updating of sub-regional groups is now automatic via a cronjob. It is documented in create-group.pl (see above)

We have given up and Disabled Falcon at the University of New South Wales? _UPDATE: Dr. _Anjum says we can have a node at Newzealand, one in Auckland. How is this going? The other node can be in Malaysia in the state of Johar in Malaysia. UNS node was managed by a student who has graduated and we can't get it better for now.

• Pinger Map, work being done on adding graphs for going back to previous year. They are zoom-able and would be sufficient to give view of a node's performance.
• Formating CSV files generated by ping-table to graphs acceptable pattern.

PingER traceroute archive site

See [[http://pinger.seecs.edu.pk/cgi-bin/traceroutearchive.cgi|http://pinger.seecs.edu.pk/cgi-bin/traceroutearchive.cgi]|http://pinger.seecs.edu.pk/cgi-bin/traceroutearchive.cgi] .. some improvements:

• Choose hosts without worrying about the date.
• Latest successful traceroute shown from A to B (left hand box) and B to A (right hand box).
• Changes (from A to B and B to A) also shown since a past successful traceroute (diff between the latest traceroute and the one half way since inception time (i.e. since April 1, 2011).
• Some sites might not be working due to PERN proxy policy. Sites that are on PERN network do not allow connections outside the 111.68.y.z IP domain. Some are just simply unavailable.
• Traceroute.pl has been updated at Pakistani monitoring hosts.
• Anjum has seen anomalies, will send output and we will fix.

PingER archive site -- FYP (Ghulam and Farhan)

• Revised the schema according to Umar's recommendations.
• Change in Nodes table of archive site database schema - included all information of nodes.cf file in Nodes table.
• Information of nodes comes in this table through NODEDETAILS table at SLAC.
• Data from NODEDETAILS table is collected by a script node.pl and we changed this script to store data in Nodes table instead of nodes.cf file.
• getdata.pl script has been changed to collect data from monitoring nodes and store it in to Ping_data table of archive site database instead of files.
• Changed getdata.pl script to collect nodes data from Nodes table instead of nodes.cf file because it will not be used in new architecture. The fields where sequence number or rtt is not present, NULL is used.
• Main analysis script is analyze-hourly.pl which executes daily on archive site and does analysis. This script has also been changed to get input for analysis from ping_data table instead of files. Tested and optimized.
• Other scripts analyze-daily.pl, analyze-monthly.pl, analyze-allmonths.pl, analyze-allyears.pl will be changed to get nodes information from nodes table instead of nodes.cf file. Remember these scripts are using the same data that analyze-hourly.pl is inserting into the analysis table.
• Port to SLAC
• Someone was going to compare the perfSONAR pinger schema with that at SEECS. (Zafar) 
• This is basically complete but yet not deployed due to possible change of schema. Once we finalize the schema, it wont take long to copy data to the database. A meeting will be setup among Dr. Anjum, Ghulam, Farhan and Bilal from SEECS and Dr. Les, Faisal and Zafar from SLAC.

Adding MOS and Alpha to pingtable.pl

• Analysis scripts to add Mean Opinion Score and Alpha, some things need to be correctly configured. It has been deployed at [http://pinger.seecs.edu.pk/cgi-bin/pingtable.pl|http://pinger.seecs.edu.pk/cgi-bin/pingtable.pl] for testing.
• MOS shouldn't be below 1, investigate 0.000 values for both MOS and Alpha. Solved. Problem arose in analyze-daily script when distance between two hosts was 0. Solved by catering for this issue in the analyze-daily script
• Alpha and MOS to be implemented at SLAC site. Sadia will be doing this with the help of Zafar
• Reason for low Alpha values: High or very high minimum RTT, which is used in Alpha's calculation.

TULIP

• With the move to the new host Faisal had to get a new cookie to allow scriptroute to run scripts on the PlanetLab infrastructure. It seems to work, or at least some PlanetLab landmarks are working. Still needs more testing and Faisal needs to contact the owner to see why so few PlanetLab landmarks work.
• TULIP is slow because of three reasons. Landmark selection is done without keeping tiering and location in mind PerfSonar landmarks are very slow.
• Les has extended the diagnostics in tulip-tuning.p. Les believes he has understood the every other day disabling of Tulip landmarks and fixed it. He has commented out the daily disabling of landmarks in trscrontab, but will re-enable when he has time to ensure the new code fixes the problem.
• We have submitted a PO to get a MatLab license for Sadia.

CBG TULIP Integration -- FYP (Bilal)

• TULIP setup on maggie2 server and CBG is running on PERN machine.
• CBG is modified to talk to TULIP. TULIP is modified for integration.
• Details of integration are [here|https://confluence.slac.stanford.edu/display/IEPM/CBG-TULIP+Integration].* TULIP code was not running due to the machine sitting behind a proxy and unavailability of a live IP. Bilal used his home system to configure this and it worked. Replicated this to the maggie2 server. This has NAT routing and proxy policy issues. NUST HQ is dubious about external TCP traffic and thus blocks this traffic. NUST HQ is being contacted to resolve or allow traffic from reflector @ SLAC.* Latest update: Bilal is using the PERN machine (same machine being used for PerfSONAR deployment) which is on a Micronet connection for deploying CBG (Matlab code). The problem he is now facing has to do with the physical link connecting the PERN machine. A network engineer found that 50% of the packets are lost over the link and those that are not have a higher RTT (?~50ms). Micronet was contacted and they have advised to check the physical link.
• Update (Bilal sent Zafar an email): TULIP is 40% accurate, Apollonius 50% and CBG 70%. He hasn't shared results yet.

IPv6 activities at NUST SEECS and at SLAC, Pakistan

PingER2 works with IPv6, could start taking data and see what breaks. Here is the list of activities we have been doing at the moment:

• Training and Awareness: We have conducted a workshop and a seminar on IPv6. Another Seminar is planned in the coming weeks.
• Collaborations: We have created a tunnel with cybernet (local ISP) for global IPv6 connectivity. We have been allocated public IPv6 addresses by PSEB for R & D usage.
• Implementation: We are currently working for participation in world IPv6 day. Some of the services that we are currently working on are Webserver, DNS and local connectivity of IPv6 for SEECS users. We are short of dedicated hardware for this part and for this our proposal is under review. We hope to have full range of hardware covered once it is approved.
• Research: We have a team of 3-4 people working on Transition Mechanisms. Open source implementation of these mechanisms.Mobility Extensions. LISP for IPv6. Hands on implementation on NS2, GNS3.
• We are setting up an isolated (vis dual homed bastion host) an IPv6 subnet at SLAC.

PerfSONAR (Pakistan)

• PerfSONAR at SEECS: Problems were fixed. NTP servers were causing considerable clock delay. Added close-by Stratum 1 NTP servers to solve the problem. Nodes were updated to PerfSONAR version 3.2 (Fedora distro). Nodes however are offline since they were disrupting normal traffic. We are waiting for 10 Mbps dedicated connection to switch the nodes back on. We have a 1 Mbps link for PerfSONAR (on temporary purposes). NUST is purchasing a 2 Mbps dedicated link from WorldCall. No progress yet .. routing issues are showing live IPs as inaccessible.
• PERN will deploy perfSONAR at HEC/Quetta. Someone is working on this. The university is close by HEC/Quetta. Hope in 4 weeks to have PingER monitoring node in 4 universities in the Quetta region.
• No Updates as Dr. Anjum were not in Pakistan for past couple of weeks. He will talk to Wahab and Imran Ashraf and get the update. Headquarters did not allow this but SEECS have some direct IP so we need to find if we can get a direct IP for this purpose, then we can deploy perfsonar.
• Anjum will let Zafar know the IP addresses

PerfSONAR (USA)

1. Zafar is working on making SNMP-MA remote-connection-capable. It will provide the ability to read RRDs over the network instead of having the RRDs on the same machine as the RRD client. Code is working and testing is complete. Yee wants me to work on a few changes from architectural perspective. Done. Committed to Internet2 svn (go under zafar at http://anonsvn.internet2.edu/svn/perfSONAR-PS/branches/ to view).

Possible projects

• See [https://confluence.slac.stanford.edu/display/IEPM/Future+Projects]. Zafar will talk to the students about these projects.
• Extend the NODEDETAILS data base to allow entry support for whether the host is currenty pingable. 
• Extend Checkdata to provide emails automatically, see [https://confluence.slac.stanford.edu/display/IEPM/Extend+checkdata+to+make+it+more+useful]. Many of the ideas in the script node-contacts.pl are a step in this direction.
• Improve the PingER2 installation procedures to make it more robust. This might be something for the person(s) in Pakistan who are responsible for installing PingER2 at the Pakistani monitoring sites. They probably have found where the failures occurs. Also look at the FAQ, and ping_data.pl which has been improved to assist in debugging, could it be further improved (e.g. provide access to the httpd.conf file so one can see if it properly configured)? There are 2 students working on the PingER archive. *Is this something they could work on?*
•  [Fix PingER archiving/analysis package to be IPv6 conformant|IEPM:Make PingER IPV6 compliant]. Will build a proposal for an IPv6 testbed. They will try various transition techniques. A proposal has been prepared and that has been submitted to PTA. Adnan is a co PI. It is being evaluated today.  A small testbed has been established in SEECS and the plan to shift some of the network to IPv6. Bilal is part of 3 students involved with PingER and they will be involved with IPv6. They are porting the PingER archive site site to using a database. They have redeveloped the archive site using Umar's documentation. They have set up a small test archive site. They have gathering, archiving, analysis. They will design a new database. They will also try a port of PingER to IPv6. 
• Look at RRD event detection based on thresholds and how to extend, maybe adding plateau algorithm. Umar's algorithm did  not work in a predictable manner. 
• Provide near realtime plots of current pinger data using getdata_all.pl/wget. It will work as a CGI script with a form to select the host, the ping size, and the time frame to plot. It will use wget or getdata_all.pl to get the relevant data and possibly RRD/smokeping to display the data. Adnan thinks one of the students working on the archive site may take this on

Paper - Umar, Fida

• SVN for paper, Umar has set it up and it is working. Umar will send out the accounts. Not critical at the moment.
• Umar will take as the main/first author of the paper. Ali Khayam has put together an introduction
• identifying the main points. Umar will send in the first draft and put together with feedback from Ali. Umar hopes to be done by Mid December. Umar is currently overwhelmed. Zero progress 11/10/2011. Umar has to submit a camera ready copy in the next couple of days.
• Adnan is also interested in working on the paper and will talk to Umar.
• Umar sent draft to Adnan. Umar will be in Pakistan later this month and will get together with Adnan. Fida will be available after December 16th.
• Fida will send review comment to Adnan
• Meeting between Adnan, Umar, Ali Khayam did not come to pass due to Adnan being sick. It is believed we need a re-write, the technical details are OK. Adnan will get to more rigorously. 
• Tulip is not working yet, also Fida and Umar are unavailable. So this paper cannot be prepared for now.

Future meeting time - Les

The next meeting in Wednesday, June 1, 2011 (8 pm) for people in US and Thursday, June 1, 2011 (8 am) for people in Pakistan.