Agenda for SEECS/SLAC meeting April 13, 2011.

Bold face, topics for coming meeting that we need to address

Future

• Sadia & Amber: Will arrive on Wednesday, April 20th. We have their itinerary and will pick them from the airport.** Amber's thesis is nearly finished. She'll be free in a day or two.
  • Sadia is also under way to finish her thesis.
• SVN possibilities? Umar set one at SEECS on maggie server. Sadia and Zafar will be working on PerfSONAR so it makes sense to use an SVN. Zafar needs to explore. This is just an idea and is not necessary if it proves to be a hindrance in usual work.

Pakistani case study - Zafar, Anjum

1. Arshad reported that the 5 new hosts were sent to five different PERN PoPs on Friday evening, last week.

   TCS Details (02 boxes each): ------------------------------------------ 1) Jamshoro Mr. Abdul Samad TCS No:500-00170225 By Air 2) Faislabad Mr. Ahsan Raza TCS No:905-1613400 By Road 3) Multan Mr. Raheel Raza TCS No:905-1629701 By Road 4) Karachi Mr. Naeem Ahmed TCS No:905-1629702 By Road 5) Quetta Mr. Rehan Sarwar TCS No:905-1629703 By Road

    27 more hosts are yet to be deployed.
2. HEC is being dissolved. Funding is about to finish, can carry people for about 1 year. Unclear what happens to the people who run the PERN network. This is really bad news for higher education in Pakistan. It is quite likely to happen. Some stuff will shift to the Ministry of Science & technolgoy. We will lose some standardization, accountability, all scholarship process. The PERN network may move to the Ministry of Information Technology. It is hard to get someone to put pressure on the university contacts, it looks like we can no longer expect HEC to help. Latest: Most probably the dissolution will not take place. Supreme Court of Pakistan has taken action against dissolution. We will have a clearer picture after a week.
3. Maybe when Anjum visits Lahore may be he can get some help for non responsive contacts, e.g. pinger.cemb.edu.pk, and lse.seecs.edu.pk which is registered in NODEDETAILS with nodename lse.seecs.edu.pk however in its pinger.xml its <SourceName> is <SrcName>111.68.102.40</SrcName>, so we need to change the <SrcName>lse.seecs.edu.pk</SrcName> in pinger.xml for lse.seecs.edu.pk## Anjum has hired two full time RAs to deploy and fix the nodes. They will be working to provide support for PingER hosts.
   1. Only possible delays would now be due to lack of support from universities or lack of will.
4. There are of lot of hosts that are failing. Why is the beacons.txt empty at SEECS? there is plenty of disk space. There was a problem with virtual memory caused by zombie processes. Some were caused by cronjobs, other by a program Zafar wrote. Zafar tried to improve the code, so far so good. The cron job was pingtable. The machine is heavily loaded. The load appears to be coming from iperf jobs. It maybe coming from iepm. Umar logged on and killed a lot of processes and stopped iepm which should help.
5. Has anyone at PERN been approached about giving someone from HEC the responsibility for ongoing support. Concern will be that once the nodes are deployed, we will have all the latest patches but since these nodes will be on internal network and it could be a potential security hazard once security is breached. Umar suggests that we should get this in writing from HEC that they'll be responsible for these nodes on their POPs. This is probably in the noise compared to the HEC problem. Anjum will send email to Anwar Amjad or one of his lieutenants  to ID someone to manage the nodes.
6. Putting monitoring hosts on UPS. Anjum is following on this with PERN.
7. Separate the analysis presentation of the PERN from non PERN connected nodes. Les emailed some information on how to do this using affinity groups. Sadia was working on this. No progress 3/30/2011.
8. Anjum was going to get an up to date map of PERN network topology. Anjum has sent a reminder to Anwar Amjad. No progress yet, Anjum will try another person, i.e.  Naveed Fayyaz at PERN.
9. PERN wants MOS further broken down by PERN and non PERN. Anwar Amjad wants an inter-city analysis. VoIP calls are relatively bad for inter-city links within Pakistan as compared to VoIP calls made outside of Pakistan (such as USA). Anwar Amjad is most interested in Islamabad to Quetta, Islamabad to Karachi and Islamabad to Lahore links. Umar mentions that it is completely fine doing this but asks whether HEC can provide us with statistics. Anjum says HEC is using fluke for measurements and they will share their data.
10. Did anyone add more information to the case study on how throughput is derived and links to justification? Zafar add a URL to metrics on case study page. Done.

PingER

• Pinger and Traceroute are working at SESAME in Jordan. It has been added to NODEDETAILS. Faisal has added it to TULIP.
• Google Form/Spread sheets may be used to reduce friction in information gathering process for new hosts. Faisal
• The Sarawak host now has the correct SrcName, but the location for Lynx is wrong.
• Sadia was working on adding alpha to prm.pl, she was going to use nodes.pl to get information on the Lat/Longs. It is still giving huge values for alpha. She will look at. It can be found at s_prm.pl
• PingER archive site: Final Year Project of Ghulam and Farhan.** Revised the schema according to Umar's recommendations.** Change in Nodes table of archive site database schema - included all information of *nodes.cf* file in Nodes table. Information of nodes comes in this table through NODEDETAILS table at SLAC.** Data from NODEDETAILS table is collected by a script node.pl and we changed this script to store data in Nodes table instead of nodes.cf file.** getdata.pl script has been changed to collect data from monitoring nodes and store it in to Ping_data table of archive site database instead of files.
  • Also changed getdata.pl script to collect nodes data from Nodes table instead of nodes.cf file because it will not be used in new architecture. The fields where sequence number or rtt is not present, NULL is used.
  • The main analysis script is analyze-hourly.pl which executes daily on archive site and does analysis. This script has also been changed to get input for analysis from ping_data table instead of files. Testing and optimizing this script.
  • The other scripts analyze-daily.pl, analyze-monthly.pl, analyze-allmonths.pl, analyze-allyears.pl will be changed to get nodes information from nodes table instead of nodes.cf file. Remember these scripts are using the same data that analyze-hourly.pl is inserting into the analysis table.
  • Changed analysis scripts to add Mean Opinion Score and Alpha, some things need to be correctly configured. It has been deployed at http://pinger.seecs.edu.pk/cgi-bin/pingtable.pl for testing.*** MOS shouldn't be below 1, investigate 0.000 values for both MOS and Alpha.
    • Once the code is in stable state and is legibly written we will be able to take the step forward to deploy it to SLAC.
• Pending until Sadia arrives here (to be discussed alongside PSI stuff):** You should compare your schema with the one used by perfSONAR for PingER. Faisal will provide Zafar with the perfSONAR schema for PingER and Zafar will compare the perfSONAR schema for pinger with his.
• PingER Map (Feature Requests)** Chrome is has something overwriting part of the tabs, for the moment use Firefox instead.** Assuming it is easy do not remove a box if one clicks on something else, leave that to be done manually by clicking on the x in the top RH corner** Add auto ranging** The ability to save a color range setting
  • Make it less browser dependent

TULIP

• With the move to the new host Faisal had to get a new cookie to allow scriptroute to run scripts on the PlanetLab infrastructure. Still needs testing.
• CBG Integration with TULIP: Final Year Project of Bilal.** TULIP and CBG were setup on one machine. Now they are setup on separate machines. TULIP is on maggie2 server and CBG is running on PERN machine.** CBG is modified to talk to TULIP. TULIP needs to be modified to complete the integration.** We changed TULIP code to write output in a file which will be used as input to the CBG code.** We transfer this file from TULIP to CBG through sockets. ** Sockets are written in perl scripts to send and receive input and output files.** Perl script at MATLAB machine reads this file and runs CBG code with required parameters.** Next step will be to have CBG code generate results and save its output to a file.** This output file is again transferred to TULIP through perl scripts (Sockets).
  • TULIP code was not running due to the machine sitting behind a proxy and unavailability of a live IP. Bilal used his home system to configure this and it worked. Replicated this to the maggie2 server.
  • Zafar installed Matlab on PERN machine with live IP and told Bilal to contact Imran Ashraf (Networks Department). Have asked for updates from Bilal (April 5th and April 10th) but no answer received. Dr. Anjum will follow up on this.

IPv6 activities at NUST SEECS, Pakistan

PingER2 works with IPv6, could start taking data and see what breaks. Here is the list of activities we have been doing at the moment:

• Training and Awareness:** We have conducted a workshop and a seminar on IPv6** Another Seminar is planned in the coming weeks.
• Collaborations:** We have created a tunnel with cybernet (local ISP) for global IPv6 connectivity.** We have been allocated public IPv6 addresses by PSEB for R & D usage.
• Implementation:** We are currently working for participation in world IPv6 day.**  Some of the services that we are currently working on are Webserver, DNS and local connectivity of IPv6 for SEECS users.** We are short of dedicated hardware for this part and for this our proposal is under review. We hope to have full range of hardware covered once it is approved.
• Research:** We have a team of 3-4 people working on Transition Mechanisms** Open source implementation of these mechanisms** Mobility Extensions** LISP for IPv6
  • Hands on implementation on NS2, GNS3

PerfSONAR

Problems were fixed. NTP servers were causing considerable clock delay. Added close-by Stratum 1 NTP servers to solve the problem. Nodes were updated to PerfSONAR version 3.2 (Fedora distro). Nodes however are offline since they were disrupting normal traffic. We are waiting for 10 Mbps dedicated connection to switch the nodes back on. - Zafar talk to Ajmal. We have a 1 Mbps link for PerfSONAR (on temporary purposes). NUST is purchasing a 2 Mbps dedicated link from WorldCall.

1. Faisal is working with Yee##  Moving XML responses to json arrays (parsing). Done## A Datagrid module to display tests from all the result set hosts for a selected

1. 1. Ability for datagrid module to share data and calls with map.
   2. Documentation available here 
   3. [https://confluence.slac.stanford.edu/display/IEPM/Google+Maps+for+PerfSONAR]
2. Zafar deployed PerfSONAR at SEECS. It was working and gathering data. However the Network Operations Center at SEECS had to shut it down since it was flooding the network and disrupting normal traffic (education/research). Sent an application to HQ NUST to approve 10 Mbps bandwidth link. The proposal was rejected. Now waiting to utilize bandwidth from a research lab's internet link. The link was available but was not setup (when Zafar left there were configuration issues). Machines are still offline.
3. PERN will deploy perfSONAR at HEC/Quetta. Someone is working on this. The university is close by HEC/Quetta. Hope in 4 weeks to have PingER monitoring node in 4 universities in the Quetta region.

Possible projects

• See [https://confluence.slac.stanford.edu/display/IEPM/Future+Projects]. Zafar will talk to the students about these projects.
• Extend the NODEDETAILS data base to allow entry support for whether the host is currenty pingable. 
• Extend Checkdata to provide emails automatically, see [https://confluence.slac.stanford.edu/display/IEPM/Extend+checkdata+to+make+it+more+useful]. Many of the ideas in the script node-contacts.pl are a step in this direction.
• Improve the PingER2 installation procedures to make it more robust. This might be something for the person(s) in Pakistan who are responsible for installing PingER2 at the Pakistani monitoring sites. They probably have found where the failures occurs. Also look at the FAQ, and ping_data.pl which has been improved to assist in debugging, could it be further improved (e.g. provide access to the httpd.conf file so one can see if it properly configured)? There are 2 students working on the PingER archive. *Is this something they could work on?*
•  [Fix PingER archiving/analysis package to be IPv6 conformant|IEPM:Make PingER IPV6 compliant]. Will build a proposal for an IPv6 testbed. They will try various transition techniques. A proposal has been prepared and that has been submitted to PTA. Adnan is a co PI. It is being evaluated today.  A small testbed has been established in SEECS and the plan to shift some of the network to IPv6. Bilal is part of 3 students involved with PingER and they will be involved with IPv6. They are porting the PingER archive site site to using a database. They have redeveloped the archive site using Umar's documentation. They have set up a small test archive site. They have gathering, archiving, analysis. They will design a new database. They will also try a port of PingER to IPv6. 
• Look at RRD event detection based on thresholds and how to extend, maybe adding plateau algorithm. Umar's algorithm did  not work in a predictable manner. 
• Provide near realtime plots of current pinger data using getdata_all.pl/wget. It will work as a CGI script with a form to select the host, the ping size, and the time frame to plot. It will use wget or getdata_all.pl to get the relevant data and possibly RRD/smokeping to display the data. Adnan thinks one of the students working on the archive site may take this on
• Provide analysis of the traceroutes in particular identifying changes -- Anjum and Zafar need to discuss issues.
• Provide better support for affinity groups

Paper - Umar, Fida

• SVN for paper, Umar has set it up and it is working. Umar will send out the accounts. Not critical at the moment.
• Umar will take as the main/first author of the paper. Ali Khayam has put together an introduction
• identifying the main points. Umar will send in the first draft and put together with feedback from Ali. Umar hopes to be done by Mid December. Umar is currently overwhelmed. Zero progress 11/10/2011. Umar has to submit a camera ready copy in the next couple of days.
• Adnan is also interested in working on the paper and will talk to Umar.
• Umar sent draft to Adnan. Umar will be in Pakistan later this month and will get together with Adnan. Fida will be available after December 16th.
• Fida will send review comment to Adnan
• Meeting between Adnan, Umar, Ali Khayam did not come to pass due to Adnan being sick. It is believed we need a re-write, the technical details are OK. Adnan will get to more rigorously. 

AOB

Future meeting time - Les

The next meeting in Wednesday April 27, 2011 (8 pm) for people in US and Thursday April 28, 2011  (8 am) for people in Pakistan.