IEPM Group Task List Current

Action Items

Terapaths Netflow (see http://iepmbw.bnl.gov/netflow/index.html) - Yee

1. Get security requirements from SLAC/John H (Yee will talk to Gary 11/14/06) - Yee
2. Presentation/Front-End - Yee
   1. Reimplement spider and pie charts for SVG::TT - Shahryar, Yee
      1. Get account at BNL - Shahryar
      2. Implement pie chart class in SVG::TT - Shahryar, Yee
      3. Implement spider class in SVG/Template format - Shahryar, Yee
      4. Tidy up legends for pie and spider charts - Shahryar, Yee

   2. Fix labels on time series plots - Yee [DONE 20061019]

3. Processing Back-End (awaits traceanal 11/21/06) - Yee, Akbar## Factorise out TopN code in JKFlow.pm (to do 11/14/06) - Akbar
   1. Refactorise JKFlow code for QoS analysis (restructure file structure and possibly QoS analysis) - Akbar
   2. Technical Documentation - Akbar
4. Testing and Installation - Yee, Akbar
   1. Finalise installation script - Akbar, Yee
      1. Add patching - need distribution mechanism for tarball storage - Yee, Akbar

      2. Test perl prefix installations - Akbar, Yee [done 20060905]

   2. Add UI/cgi code to installation - Akbar, Yee
   3. Try out installation on fresh machine at SLAC

Transport services evaluation - Yee

1. Work with Microsoft - Yee, Les
   1. Get latest privates from Microsoft
   2. Decide what is needed for stage 2
   3. Install Vista/Longhorn (Stalled: Windows Vista does not install 11/14/06) - Yee
2. NDT server at SLAC (see http://nettest5:7123) - Yee

   1. Installed on nettest5 NDT Install - Yee [DONE 20060831]

   2. Talk security into allowing public access - Yee.

PingER

1. PingER Visualization, Shahryar
   1. Allow identification of nodes in top left hand corner (was done 8/3/06, but has come back 8/7/06, re-reported 11/15/06): Shahryar
   2. Plot multi metrics (e.g. RTT & loss, agreed 6/7/06): Shahryar
   3. Zoom has a bug if one shows links from SLAC to World (reported 11/15/06) : Shahryar
2. PingER2 improvements
   1. Provide multiple sources for beacons file: Warren
   2. Add diagnostic for missing beacons.txt: Warren
   3. Add diagnostic for failed lookup: Warren
   4. Add FAQs on missing beacons and failed lookup - Les
3. Review beacons (make sure SLAC, S. Africa, Bolivia, Sao Paolo, NIIT, ICT, CERN can see all Beacons, there are some beacons e.g. lnfnet.lnf.infn.it that are pingable from SLAC but not other places): - Jerrod
4. Convert offsite.nodes to a Guthrie database: Jerrod
   1. Add contact names etc. (Done,  currently populating list 10/15/06 - Jerrod)
   2. Enable entering the Group (Jerrod is working on 7/28/06, no update 10/16/06): Jerrod

Redesign and Implement Guthrie to cover both IEPM and PingER

IEPM-BW

1. Work with DESY & NIIT to get monitoring hosts up  (Jerrod will read Connie' documentation and see if he can accomplish this, adding to the documentation as he goes along, reading/understanding documentation ) - Jerrod
2. Make RAL a remote node
   1. Have account but cannot ssh to it (sent email to Tasker 9/8/06, await PingER to run) - Les

3. Add LHC Atlas hosts to IEPM-BW (list sent to Jerrod 9/6/06, added needs testing 9/6/06) [Done 9/8/06] - Jerrod, Les, Connie

4. Add group for US-ATLAS [Done 9/20/06] -  Connie

5. Toolspecs for Thrulay multi stream (includes selecting good windows/streams) - Jerrod, Connie
6. Get architecture of remote nodes and create a web page - Jerrod
7. Write script to use ssh to get the configurations of IEPM monitor and remote hosts (in progress 4/26/06, will revisit Sep '06, possible project ) - Jerrod
   1. Report on anomalous values - Jerrod
   2. Add Nagios/Ganglia?
8. Do we want to get reverse traceroutes (at least where we have reverse traceroute servers, awaits time) - Connie
9. Student project to make automate installation for scripts for IEPM-BW remote and monitoring sites -  Les,  Yee
10. Prepare table of canonical events and how various algorithms react - Adnan
11. Build case studies of email event (see Anomaly+Case+Studies)  - Adnan
12. Put together architecture document that explains how the scheduling, analysis stuff works - Connie
    1. Document how to add probes (follow on from mthrulay) - Connie
13. Bugs:
    1. BNL is very slow, maybe a swapping problem (only about 16MB free on 2GB host)
    2. Possible conflict with Netflow
    3. Reduce memory requirements of IEPM-BW?

Traceanal Modularisation - Yee, Asif

1. Integrate new topology into web server - awaits wan-mon appropoval - Yee
2. Rendering of topology much slower on www.slac.stanford.edu - likely to be related to the web server, not code, as it runs quickly elsewhere (awaits WANMON web server, see below) - Yee
3. Compress trcaeanal table - Nouman, Asif
4. Add node name in RH column
5. Make nodename clickable to view ping time-series
6. Add comment or help on how to find the real hostname
7. See how well the traceroute analysis does on monitor.niit.edu.pk - Les
8. Add color to Graphviz edges - Akbar
9. Add end node even if it does not respond
10. Traceroute_analysis - Asif, Yee
    1. Prepare distributable version of traceanal - Asif, Yee

    2. Tidy up presentation - Asif [DONE 20061116: cannot be done more smaller using CSS]

    3. Profile code for speed improvements - Asif, Yee [DONE 20061117: using dProf module]

    4. Provide example of how to allow integration into Non-IEPM-BW data sets - Asif, Yee

    5. Update all Test_cases -Asif, Yee [DONE 20061117: All availabe test cases are updated with the current implementation]

IEPM-BW Ping Visualization - Les, Asif

1. Update Traceanal Modular code to gather and visualize IEPM ping data - Asif [BASE Code DONE 20061116: Rest will be assigned to student at NIIT], see compress traceanal table

2. Change title for ping table output
3. Add mouseover help for meanings of colors etc
4. Look at alerting outages for ping 

Alerts and Diagnosis - Les, Yee

1. Step changes (Brian Tierney LBL is interested in predictions): Waqar, Adnan
   1. Yee is writing a new version of Plateau (11/16/06)
2. Look at multivariate event detection (collect data for SLAC, BNL, Caltech pathchirp, thrulay,ping) - Adnan
3. Look at improvements to plateau (Yee plans a rewrite, Akbar interested)
   1. Ability to find step ups - Yee
   2. Extend to allow up & down then compare down with original - Adnan
   3. Allow for small number of samples (e.g. at start) -
4. Look at other detection algorithms and compare - Akbar
   1. Holt-Winters - Les, Yee
   2. Go back 7 weeks
   3. Check unusual results
   4. Consider other ways to optimize parameters
   5. Neural networks, Bayesian, ARMA/ARIMA ...
      1.  
5. Understand cause of delayed alerts and see if can improve - Connie - fixed - done Connie

Install WANMON as IEPM web server - Yee

1. Get traceroute.pl and pingtable.pl working and in production (requested script to be in cgi-wrap.list 11/20/06) - Les

PerfSONAR - Yee

1. IEPM-BW Web Services - Yee, Asif
   1. SQL MA in Java - Asif
      1. Install SQL-MA* info*- Asif, Yee
      2. Write Ibatis configs for IEPM-BW data - Asif

2. Measurement Archive Perl Service for IEPM-BW data - Yee [DONE 20061110]

   1. Rewrite for NetRadar - Yee
3. Java Topology Representation

   1. API interface - Asif [DONE 20061121]

   2. Embed into perfSONAR UI - Asif
4. NMWG requirements? - Asif, Yee

Presentations/Talks/Visits/Papers/Documentation

IPv6