Time & date

Wednesday 6 March  2013 8:00pm Pacific Standard Time, Thursday March  7 2013 9:00am Pakistan time, Thursday March 7 2013 12:00 noon Malaysian time.

Attendees

Invitees: Anjum*, Kashif-*, Johari23Johari*, Nara?, Abdullah?*, Badrul*, Hanan?, Fidah12Fidah, Saqib*, Les*, Raja*-, Umar * and Bebo*  

There were 3 choices:

1. Leave the meeting time as is
2. Move to Thursday 21 February 2013 7:30pm Pacific Daylight Time, Friday 22 February 2013 8:30am Pakistan time, Friday 22 February 2013 11:30am Malaysian time. Note the time change to accommodate prayers in Malaysia
3. Move to Wednesday 27 February 2013 8:00pm Pacific Daylight Time, Thursday 28 February 2013 9:00am Pakistan time, Thursday 28 February 2013 12:00 noon Malaysian time.

I would prefer it if Johari/UNIMAS can attend. My preference is choice 3. However, I can make all 3 choices.

* can attend all 3 choices.
12 can attend choices 1 and 2
23 can attend choices 2 and 3
13 can attend choices 1 and 3
? Unable to attend any of the choices
- No response

Chosen time: 

Meeting time & date:  Thursday 21 February 2013 7:30pm Pacific Daylight Time, Frida 22 February 2013 8:30am Pakistan time, Friday February 2013 11:30 am Malaysian time.

Actual attendees:

General

-  

* Confirmed attendance

- Unable to attend

Actual attendees: Badrul, Saqib, Les

General

Johari has set Johari is setting up a web landing site for the Malaysian collaboration.  The start is It  is at http://pinger.unimas.my/pinger. It will contain It contains:

• brief history, description of the project and the objectives
• link to main SLAC page and also NUSTpage 

It will later also contain

• Link to NUST site
• list of names, contact details (email, etc), their affiliation, and they the scope of job in the project
• a link to the data collected from UNIMAS and also other hosts in UM, USM, and UTM when they have their monitoring host running
• link to meeting minutes in Confluence
• we need to add Bebo added as an advisor

...

• details of the workshop in UNIMAS and also slides from the talks and hands on sessions

It would help to have an MoU between SLAC and the UTM. Les has sent a request to his legal folks to ask how to proceed and whether he can share the UNIMAS MoU with UTM and UM.  he hopes to hear back in the coming 10 days. We will also need agreement to share from UNIMAS.

Workshops

...

UTM. Based on DOE current MOU policy, using the UNIMAS MOU format will not work. SLAC's legal folks also do not think that merely adding additional parties to the current MOU will work either. The new DOE policy will require a new approval. We may be able to get DOE approval of a Work for Others agreement. To do a WFO agreement we would need, at a minimum:

1. A detailed project description describing the activities each party will perform
2. A description of the deliverables SLAC will provide
3. A schedule with completion dates for the activities
4. The key personnel and contact information for both parties
5. The term for performance of the project activities
6. An explanation of the unique capabilities SLAC will provide to the project not available elsewhere
7. An explanation how the project is compatible with existing SLAC program(s)
8. An explanation how the project enhances the mission of SLAC

Assuming we can get these issues addressed, our legal folks think we could seek DOE approval and enter into an agreement with these other universities. We need to hear back from UTM and UM whether we need/want to pursue this further. It will probably be a considerable effort. UTM & UM will need to address 1. We all need to think about 3. 4 is easy,  2 is doable, 7 and 8 are tricky.

Workshops

Johari recommended a future PingER workshop later this year. This time it would be in a Peninsular Malaysia site hosted at either UTM or UM. Hanan at UTM has agreed to take the lead.  We need to get 4 or 5 universities involved. We will need travel funding to fund Les. Les has provided his CV to assist in the paper work. This may be associated with a thesis review or something similar. Update Hanan?

We need to set some dates. Anjum prefers it to be out of semester time but that is not critical. Les has a constraint  starting July 12th through August 4th. Hanan may have a constraint early June.  Malaysia has a lot of public holidayholidays. What is the period that we are looking at? (June - July?), .

 A A quick list: 10th July to 8th August is fasting month (Ramadhan); 25.05.2013 - 09.06.2013  School Holiday in Malaysia. 

Apart from that, if we target a lot of academicians, then the best time would be during semester break, we will get the dates for that.

*Hanan, we need to get moving on this. Should we set up a Doodle page, see http://doodle.com/?locale=en (http://doodle.com/?locale=en*. )?

Funding

Johari has submitted a proposal for FRGS (Fundamental Research Grant Scheme). There are 5 members of the proposal (5 is the max). Johari decided to get 2 more people from the group in UNIMAS to join the PingER project, thus we are unable to include our counterparts in UM and UTM (Dr Abdullah and Prof Hanan). We can go for a bigger project such as the eScience or LRGS (Long-term Research Grant Scheme) which can accommodate a larger number of people, more funding and encourage cross institutions collaboration. Any Update Johari?

At a later stage the infrastructure and data can be used for research in: anomaly detection; correlation of performance across multiple routes; and for GeoLocation. Johari would like to see UM and UTM, which are research universities, strongly engaged in the next round of research proposals. Anjum mentioned a paper on Evaluation of IP Geolocation Algorithms on PingER and PlanetLab Infrastructures, by Fida Gilani et. al. submitted to IEEE INFOCOM 2011UNIMAS. This could be updated with new data and maybe submitted to a different venue. Anjum forwarded a copy of the paper to Johari, Abdullah and Hanan with copies to the team. The idea is to see whether there is interest at UTM, UM or UNIMAS. Johari has put together a draft that has been reviewed by Anjum. It will need to be shared when closer to completion. Update Johari?

UNIMAS

There is a problem with the UNIMAS traceroute. It appears pinger.unimas.my is behind a NAT that is not passing the traceroute UDP probes. Other hosts at UNIMAS are not seeing the problem, so Johari believes it may have to do with the IPTABLES in pinger.unimas.my. Traceroute using the -I option (send the probes using ICMP rather than UDP) seems to work (but needs root access). Thus it would appear not to be in the return (ICMP "time exceeded") response. The probes (when not using the -I option are UDP packets using sequentially ports 33434 - 33465 for up to 30 hops). It is possible that ports > some number may be blocked. Update Johari, Anjum.

Johari has set up a guest account for Anjum. Anjum logged on and reported some concerns about addresses and interfaces not matching. The traceroute/ping server is important for the TULIP trilateration project and also understanding routing within and from Malaysia.

Johari was out of town and had limited Internet connection until 21st feb 2013. Also he realized that the ssh and ftp server is not running since the previous reboot, thus he was not able to make any changes to the UNIMAS website until he gets back and manually start those two services.

UTM

Saqib has successfully installed pinger2.pl at UTM. It is working and we are successfully gathering data from it.  <HostList>  has been installed and it is successfully running the latest version of pinger2.pl..

UTM

The UTM Pinger site The URL is http://161pinger.139fsksm.146utm.158my/cgi-bin/ping_data.pl. The host's name (pinger.fsksm.utm.my) is registered in DNS as of 2/14/2013 (a Valentine for PingER) and is in the PingER NODEDETAILS meta datbase to get the data. .  

The traceroute server is also running. It is at: http://pinger.fsksm.utm.my/cgi-bin/traceroute.pl, however it is not working, The ping server, i.e. http://pinger.fsksm.utm.my/cgi-bin/traceroute.pl?function=ping is working.

traceroute -I www,cern.ch works which suggest the UDP probes to UDP ports 33434 - 33465 are being blocked.  Saqib is working with the IT department to understand. The output is different from the web browser and the command line. The command line using traceroute www.cern.ch looks fine. The web server stops after the 4th hop.  The web server issues the command traceroute -m 30 -q 3 137.138.144.168. The m is  the max number of hops,  the q is the number of probes per hop, 137.138.144.168 is the IP address of www.cern.ch. This also works from the command line. It could be a timeout problem (it would be good to time traceroute -m 30 -q 3 137.138.144.168 command).  Also look in the webserver log for possible failure causes.

Anjum has postulated that there may a load balancer induced problem.

The addresses assigned to the servers are:
physical ip : 161.139.68.188
external ip 1 : 161.139.146.158
external ip 2 : 161.139.194.158

. Saqib discovered the problem was a timeout in traceroute.pl. Saqib has extended the timeout and it works.  there may still be timeout problems and Saqib will experiment with longer timeouts of up to 280ms.

Saqib is starting to look at Pingtable.pl anomalies as a start in a case study for Malaysia and S.E Asia. Progress?. The general idea of the case study is to be found at: https://confluence.slac.stanford.edu/display/IEPM/Malaysian+Case+study. Saqib is making up a list of hosts in Malaysian states that respond to pings. 

We should discuss:

It is not easy to find such sites in those states. Obvious targets are universities, for example UUM in Kedah and UNIMAP in Perlis, but those sites are not ping-able. One way is to approach these universities and explain to them about PingER project and request for the ping packet to be enable. Les has sent HostFinder.pl to Johari to possibly assist in finding hosts.

UM

Dr Nor Badrul Anuar Jumaat has installedl PingER on a cloud at UM. The machines are virtualized and  running Ubuntu Linux. Anjum pointed out that at NUST they are having a problem with a virtual machine running pinger2.pl and the /tmp/ disk space filling up. Anjum is exploring this in more detail, so at the moment it is just a head's up. Johari has sent Kashif's workshop presentations to Dr. Nor.   UTM & UM will need to address 1  The name and address of the server is  pinger.fsktm.um.edu.my (202.185.107.238). The UM monitoring host has been added to the NODELIST PingER meta data base at SLAC a nd and the <HostList> for Malaysian and S.E Asian hosts has been added to the pinger.xml file at UM.  We We need to get the latest pinger2.pl installed. Since the traceroute is working, "we" can start to make a routing study similar to https://confluence.slac.stanford.edu/display/IEPM/Routing+from+UNIMAS.

USM

Les met with Navaneethan Arjuman of USM at the TIP meeting in Hawaii.  He seems interested in the research side of things. He is a researcher not an operations person and has been involved in network monitoring for 4 years. He talked of a tool they have developed called INFMON, if I heard right. He also is interested in a research proposal. He also strongly recommends we engage with MYREN. Les tried pinging www.usm.my but pings are blocked. He can ping nav6.usm.my. This host has been added to the hosts monitored from SLAC. Nava has provided an IPv6 address to monitor, however, SLAC has not enabled IPv6 yet.Badrul has a student who will be working on the case study and will coordinate with Saqib at UTM and Johari at UNIMAS to combine results and to put together a paper for the Malaysian ISI Journal probably around the end of June

NUST

From Arshad: Rector has agreed to support one person visit to SLAC. Raja has been nominated. He has sent a cv to Les and has filled out the DoE form. Les crafted the invitation and it has been sent to upper management for signature Dec 21, 2012.Les  briefed the SLAC Cybersecurity person on 1/10/2013. He has also filled out another form for hosting visitors from sensitive nations. Les has taken and passed the training on hosting visitors from sensitive nations. Cybersecurity want more details on what Raja can access and Les has provided it. SLAC CyberSecurity say they do not approve or disapprove visitors so I have requested (2/8/2013) the CIO to sign and pass to the CEO for signature. Raja has completed his paperwork and is ready to submit. As soon as Les hears the letter has been signed by the CIO (and before it is sent to Raja), Les will get back to Raja to proceed. He has sent 2 reminders to the cybersecrity person and engaged the CIO in the latest.  Cybersecurity wanted a rsiks and  mitigations document. It has been created and accepted. Raja's invitation letter has been signed by the CIO. It was sent to the Associate lab Director last Friday.  It was sent to Raja  just before the meeting.

Arshad and the NUST Rector visited UM and UTM earlier this week. They visited UM Monday at about 10:30AM along with the NUST Pakistan delegation. They visited the Deputy Vice Chancellor and the EE Dept in Faculty of Engineering. At this occasion they also sent email to Abdullah, Fidah, Badrul, and Salwa, to engage in a discussion on  collaboration in the PingER project being pursued by NUST team in partnership with Stanford Linear Accelerator Center (SLAC) USA.  Badrul was unable to meet due to the short notice and a prior engagement. The visit to UTM was postponed due to the NUST rector getting sick.

Any feed back - Anjum?

Kashif reports that the  Multan PoP node was under observation for many days. Now its working fine with little bit of un-reachability. It has been added back to the list of Monitors in the meta database.

Potential projects

See list of Projects

Ipv6

With Malaysia moving quickly to IPv6 (already Nava at USM has sent Les an IPv6 address that he recommends monitoring), converting PingER (especially the gathering, analysis) work with IPv6 is an interesting problem. Anjum believes he could provide an undergraduate to start on this. SLAC looks like it may be getting some traction to enable IPv6 on a some subnets. We have an official Provider Indpendent IPV6 address space and are putting together a small project. We will start with DNS, the site web server and email.

Bebo's suggestions

(1) Creative visualization of PINGer data including rich interaction;

...

With PingER going back over a decade, its presentation tools tend to be a  bit jaded having in some cases been developed in the 90's. New, modern ways to access display and navigate the data would be a  big plus.

Coordinate and lead the installation of PingER monitoring at multiple Malayisan sites

Build and install a new http://www-iepm.slac.stanford.edu/pinger/tools/pingER-2.0.2.tar.gz file with the improved version of pinger2.pl

PingER at SLAC

The annual report to the International Committee for Future Accelerators (ICFA) Standing Committee on Inter-regional Connectivity (SCIC) has also been completed. Les has been invited to give a general colloquiam at SLAC on the Internet where it came from, challenges and how it is performing. the latter section will report some PingER case studies and measurements. 

TULIP

Raja has completed the MATLAB script and used it to generate some input files for Pakistan and Europe. However using the alpha values (function of both region and RTT value) found through the alpha analysis the distances were underestimates in some cases. He is working on finding the cause and possible solution.

Future meeting  - Les

Next meeting Wednesday ? meeting Wednesday 20 March  2013 8:00pm Pacific Daylight Time, Thursday ? 21 March 2013 9:00am Pakistan time, Thursday ? 21 March 2013 12:00 noon Malaysian time.

Future agenda items:

Workshop: dates, funding, other sites

...

Case studies: routing, performance for Malaysia and S.E. Asia. We need someone to take the lead on this for Malaysia.

Coordinates: