Maggie-NS Tasklist
Last Updated : November 16, 2006

NEXT MEETING: 9:00 AM California, USA Time, Thursday November 16th, 2006

Team members: Abdullah, Arshad, Asif, Asma, Azher, Faran, Fareena, Rabail, Shahryar, Umar

Adnan, Akbar, Connie, Jerrod, Les, Yee

Mailing lists: maggie-niit@ns.niit.edu.pk; backup: maggie-niit@googlegroups.com

Awaits something, also provides start of wait date
Done or drop, remove < 2 months are task done
: Person doing the work
Bold face: task in progress, or for discussion at meeting
Recent progress/discussion. Need student to be assigned. Critical.

Action Items

  1. PingER contacts: Akbar Mehdi, Umar Khan
    1. Provide tools to analyze and create plots of PingER data: Akbar, Les
      1. Tie into webservices/SQL access to data to make a much more flexible distributed way of selecting analyzing and displaying data
      2. Regional plots for executives  using JFreeChart: Akbar
        1. Enable choosing symbol (Akbar will look to see if easily done, else drop 11/13/06): Akbar
        2. Fix fitting for all years: Akbar(Done 15th Nov 2006)
      3. Extend prm.pl
        1. Provide normalized throughputs: Shahryar (Done: 1-Nov-2006)
        2. Add RTT/throughput/min_RTT/jiiter for continent to prm.conf: Shahryar
        3. Select filtered data [Done 11/13/06]: Shahryar
    2. PingER web services: ?
    3. Pinger Management project: Jerrod, Les
      1. Down site implementation, i.e. auto discover and report down sites : Adnan, Les
      2. Step changes (Brian Tierney LBL is interested in predictions): Waqar, Adnan
        1. Akbar is looking at this
          1. Speed up plateau by doing in 'C' [Drop 9/1/06]: Akbar
      3. Clean up data: Jerrod, Les
      4. Review beacons:(done 10/16/06 - Jerrod
      5. Convert offsite.nodes to a Guthrie database: Jerrod
        1. Add contact names etc. (Done,  currently populating list 10/15/06 - Jerrod)
        2. Enable entering the Group (Jerrod is working on 7/28/06, no update 10/16/06): Jerrod
      6. Look at anomalies for various metrics by region and country, define project: Akbar
    4. PingER Visualization, Shahryar
      1. Provide a visualroute capability [Drop 11/15/06]: Shahryar
      2. Allow identification of nodes in top left hand corner (was done 8/3/06, but has come back 8/7/06, re-reported 11/15.06): Shahryar
      3. Plot multi metrics (e.g. RTT & loss, agreed 6/7/06): Shahryar
      4. Use smokeping plots for variances and possibly uptime (awaits PDA) : Shahryar, Asma
      5. Extend to IEPM: Shahryar
      6. Zoom has a bug if one shows links from SLAC to World (reported 11/15/06) : Shahryar
    5. PingER & AMP Integration (AMP/NLANR has run out of funding, AMP will shut down 8/31/06): Abdullah, Adnan
      1. List of IP to be prepared and sent to Abdullah [Done for NIIT 6/6/06]: Les, Jerrod, Adnan
      2. Extend the addresses to add more source nodes: Abdullah
      3. Code to be sent by Abdullah to SLAC [Done 6/6/06, it converts data from a single AMP day]: Abdullah
      4. Install & integrate code at SLAC: Abdullah
      5. Improve code, make production quality, automate getting data (requested with suggestions 8/20/06, making progress hope to be done in 3-4 days 8/31/06, sent back to SLAC, SLAC responded further formatting etc. improvemnts needed 9/12/06): Abdullah
      6. Install at SLAC & integrate: Abdullah
      7. Get account: Abdullah
      8. Integrate with pingtable.pl: Abdullah, Adnan
        1. Create AMP file directory: Jerrod
        2. Set up daily crontab:  Jerrod
        3. Set up analyze-hourly etc: (Done)Jerrod
        4. Add src & dst amp nodes to Guthrie: Jerrod
        5. Think about retrieving older data (find out how long ago there is data for): Jerrod, Adnan
        6. AMP and Pinger data comparison: Abdullah
    6. Smokeping & PingER Integration: Asma (Asma plans to move on in coming months, need to complete current tasks 8/3/106), Yee
      1. Look at how to input PingER data to Smokeping display module (sounds like we have to convert from PingER to RRD format, can this be done in realtime or do we need an RRD database too): Asma
      2. Look at how to to take recent smokeping data and convert to PingER format (in progress 7/19/06, needs pingtable.pl to get all 12 metrics)
      3. Install on pinger2.niit.edu.pk (authentication issue fixed having problems with adding one day 8/31/06) - Asma
      4. Change from 90 to 120 days - Asma
      5. Integrate with Pingtable.pl (Jerrod will send pingtable.pl links 8/31/06): Asma
      6. Look at adding visualization of uptime: Asma
      7. Look at how to dynamically change the config file for Smokeping to accomodate the step interval (maybe time consuming, may affect all users and be cumbersome): Asma
      8. Look at using Smokeping to determine the quality of home links in Pakistan: Asma
      9. Will use as a top level access to PIngER and also as links from Pingtable to display last 45-90 days using smokeping: Asma, Yee
    7. Provide real-time display of PingER data:
      1. Export data to MonALISA from SLAC monitoring site:
      2. Figure how to deal with non regular (non exact 30 min data)
      3. Do for SLAC
        1. Decide on how to extend to other monitoring sites, try with NIIT: Akbar, Umer
        2. Hassan is looking into the code sent by Fawad (next step is to get ML web services client to get data): Akbar, Hassan
    8. Monitoring Hosts:
      1. NTC hosts, they agreed to install PingER2 at their 3 backbone sites
        (sent email to NTC 3/7/06 (Aziz sent email to Wajid Ali for Lahore & Karachi 3/14/06, they have asked for a physical host from NIIT, Azher will provide 4/20/06, NTC Lahore is 2nd in Azher's priority, NTC Karachi is 3rd 7/28/06, discussed with Arshad will provide host in a month 8/17/06, Azher needs to coordinate sending machine to remote location 8/31/06): Azher; (no update as of 10/18/06 - Jerrod)
    9. PingER2 improvements
      1. Provide multiple sources for beacons file: Warren
  2. Triangulate position of an unknown host (see http://www.slac.stanford.edu/comp/net/wan-mon/tulip,Faran back from ITCN 8/21/06): Faran
    1. Deployment: Faran
      1. Provide a program API: Faran, Shahryar
    2. Consider the security aspects of TULIP: Faran
      1. Maybe develop as servlet or CGI script: Faran
    3. Visualization
      1. Make tables sortable (working on 5/25/06, re-agreed 8/3/06, re-requested 8/21/06, done for the RTTs 8/22/06, can we do with buttons for all columns): Faran
      2. Can we add AS's to routes: Yee, Faran
      3. Integrate with Google maps: Faran
      4. When zooming maintain size of dots (is there a zooming feature 11/15/06): Faran
    4. Landmarks
      1. Get more Traceroute Sites to TULIP:
        1. Send email to Alaska (5/7/06): Les
        2. Send email to Singapore (got response 5/3/06, sent reminder 5/24/06, got response 5/25/06, sent email 7/26/06): Les
        3. Send email to Malaysia (5/7/06): Les
        4. Sent email Latin America (5/7/06): Les
        5. Sent email for China contacts (7/26/06): Les+#### COMTEC, Japan(Faran send email reminder 5/25/06, agreed to send another reminder 6/7/06): Faran, Farooq
      2. Add NetDiGi route servers (asked for 8/30/06): Faran
      3. Add Abilene Looking Glasses (asked for 8/30/06): Faran
      4. Add traceroute/Looking Glass servers in critical areas (asked for 11/15/06): Faran
      5. Add daily checks whether existing landmarks are working and remove if not (requested 11/14/06): Faran
      6. Use PlanetLab for landmarks
        1. Join PlanetLab: Faran
        2. Provide a daemon for the ping server: Faran
        3. Provide a client for the TULIP: Faran
      7. Method for updating the landmarks: Faran
        1. Faran has implemented an edit function but unable to make it work at SLAC (5/25/06, demonstrated to Faran 6/21/06, Akbar sent email that modification is not working 7/25/06, Faran said retry but still not working 7/26/06, Given taht this does not work Les recommends we keep the master at SLAC and drop this item): Faran
      8. Landmarks:
        1. pinger.ntc.net.pk denies accesss (email to Azher & Faran 7/25/06, Azher responded that he is checking 7/27/06, email sent again 8/21/06, Azher knows about will send person to repair 8/22/06): Faran/Azher
    5. Apply to PingER node database to check the lat/longs are roughly correct (tried with 300-400 sites but cannnot get results due to fragile Internet at NIIT, will interact with Akbar to make tests at SLAC 8/31/06, exchanged emails on list of PingER hosts to locate 9/3/06): Faran
    6. Analysis
      1. Literature search on triangulation, multi-lateration: Faran
      2. Document host names/addresses with multiple locations (e.g. root name servers) [Done 9/9/06]: Les
      3. Compare with DNS LOC records, plus availability of DNS LOCs: Faran
      4. Compare with IP2Location and/or Netgeo records (from http://netgeo.caida.org/aslatlong.txt) [Done 10/19/06]: Faran
      5. Compare with GeoIP: Faran
      6. Measure landmarks from landmarks: Faran
      7. Document Faran's algorithm, try new algorithm with optimized alpha that does not require destination to be inside triangle: Faran, Umar
      8. Compare Faran's algorithm with Bamba's, Octant & simple triangulation
        1. Contact Bamba to find out if http://planetlab-01.ipv6.lip6.fr:10000/cbg.php works for any sites (email send 4/21/06): Aziz
        2. Confidence estimates
        3. Applied to PingER hosts
        4. Outside triangles, area coverage
    7. Analysis of Accuracy and applicability
      1. Accuracy vs known locations
      2. Effect of removing, adding landmarks, try for Bolivia:Faran
      3. Comparison with other sets of landmarks & methods (can't do Bamba since it is down), Octant, NetGeo for accuracy coverage etc.
      4. Provide bubble maps of accuracy: Faran
      5. Review difference between Guthrie and TULIP
        1. For US using Octant: Faran
        2. Look at big discrepancies: Faran
        3. Compare with GeoIP: Faran
      6. What area/population is covered by TULIP
      7. Document anomalies  (e.g. root-servers): Faran
      8. Show accuracy by region and country: Faran
  3. NETRAT, SLAC contact: Adnan, Les
    1. Evaluate PCA multivariate method of Crovella: Waqar, Adnan,
      1. Compare PCA results with Holt-Winters/plateau: Adnan
        1. Fetch Data From SLAC, BNL, CALTECH (Pathchirp, Thrulay, Ping): Adnan
        2. Fetch data from CERN
        3. Apply PCA on same data and to get potential alerts: Adnan
        4. Comparison of Run-period results and PCA results: Adnan
        5. Improve event detector: Adnan
        6. Docment findings on PCA: Adnan
    2. Forecasting comparison of Active vs. passive data: Fareena, Adnan
      1. Look at eview implementation of ARIMA (QU has copy): Fareena
        1. Extend to more sites/metrics, see need for Integration & higher orders: Farena, Adnan
        2. Try with longer time base: Fareena
        3. Get time estimates : Fareena
      2. Detailed Graphs description: Fareena
      3. Regularization
      4. Look at moodss (http://jfontain.free.fr/statistics.htm): Fareena
      5. Send results/documentation to SLAC (will send 8/31/06): Fareena
      6. Automate procedures
        1. Define project
        2. Put together presentation for workshop (for Sep 2006): Fareena
        3. Gather team
    3. Automatic event isolation: Adnan, Yee
      1. Canonical Data set: Adnan
        1. Fetch data from monitoring nodes (Pathchirp, Thrulay, Ping, Traceroute): Adnan, Ali
        2. Study/Analyze Data: Adnan, Ali
        3. Run Run-period: Adnan
      2. ALERT table changes
        1. New table design: Adnan
        2. Script to populate the table: Adnan
        3. database connectivity problem: Jerrod
        4. Modify other scripts accordingly: Connie
  4. IEPM, SLAC contact Connie
    1. Install OWAMP at NIIT, get NTP working well (NTP working according to Azher 4/20/06): Connie, Azher
    2. IEPM-BW Web-services:
      1. Waiting for a new student: Umar, Shahryar Fahad Satti?
  5. Pathneck (see http://maggie.niit.edu.pk/website/projects/Topological_Analysis.html) Yee is the lead at the SLAC end: Asif
    1. Try and get pathneck working from NIIT, study debug output, work with Ningning <hnn@cs.cmu.edu> (awaits site upgrade): Asif
    2. Develop and install pathneck reverse server (would like to start at NIIT just to assure code works work with -y option): Les, Asif
      1. Modularize security of traceroute.pl: Asif, Yee
      2. Add in PerfSonar: Asif, Yee
    3. Evaluate and understand how pathneck needs to be configured, the failure modes, the areas of applicability etc. using iepm-resp.slac.stanford.edu: Asif
      1. Provide web site documenting failure modes: Asif
    4. Develop a vizualization
      1. Topology (currently with Touchgraph), color links, mouseover ID router, provide snapshot
      2. Click on link/router to get graph
      3. Incorporate location (e.g. via TULIP)
    5. Investigate integration with traceanal and graphviz: Asif
    6. Integrate topology map with pathneck: Asif
    7. Look at using PerfSONAR router information (do not agree well, need to get more statistics, to enable multiple measures need API to get PerfSONAR data, look perfsonar UI): Asif
    8. Try on Ultralight (problems if < 4 hops): Asif
    9. Problem of 0 results (reported to author but not much happening 4/20/06): Asif
    10. Add position and AS information to router database: Asif
  6. Communications etc.
    1. Projects (see http://www-iepm.slac.stanford.edu/pinger/projects/)
      1. Two possible new students are: MNouman and Fahad
    2. Proposals
      1. Submitted NTC proposal (sent for review by PTCL 8/3/06 ): Arshad
      2. Pakistan US Cooperative Grants Program 2006 (submitted awaiting review, 8/3/06)
      3. HEC solicitation for International collaborations, due mid-Sept, Farooq PI (submitted 9/17/06, await resulst)
  7. Wiki
    1. Get accounts/passwords for NIIT students - Yee
    2. Create space for NIIT student reports and populate with initial reports - Yee, Les
  • No labels