You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 40 Next »

Maggie-NS Tasklist
Last Updated : August 21, 2006

NEXT MEETING: 9:00 AM California, USA Time, Thursday September 31st, 2006

Team members: Abdullah, Arshad, Asif, Asma, Azher, Faran, Fareena, Rabail, Shahryar, Umar

Adnan, Akbar, Connie, Jerrod, Les, Yee

Mailing lists: maggie-niit@ns.niit.edu.pk; backup: maggie-niit@googlegroups.com

Awaits something, also provides start of wait date
Done or drop, remove < 2 months are task done
: Person doing the work
Bold face: task in progress, or for discussion at meeting
Recent progress/discussion. Need student to be assigned. Critical.

Action Items

  1. PingER contacts: Akbar Mehdi, Umar Khan

    1. Remove regions East Europe [Done 7/13/06]: Les, Jerrod

    2. Provide tools to analyze and create plots of PingER data: Akbar, Les
      1. Tie into webservices/SQL access to data to make a much more flexible distributed way of selecting analyzing and displaying data
      2. Regional plots for executives  using JFreeChart: Akbar
        1. Extend to allow allyears: Akbar
        2. Create a web page which displays different pinger executive graphs. (has a prototype 4/18/06, need to test): Akbar
          1. Enter should be same as Apply changes in Line patterns, also allow selection of colors: Akbar
          2. Provide cronjob capability (need to understand what is wanted): Akbar

      3. Provide plots of yearly loss, RTT and throughput (i.e. one point/year, need to get more accurate loss measures and thus derived throughput for excellent paths) [Done 9/1/06]: Les, Aziz

    3. PingER web services: ?
    4. Pinger Management project: Jerrod, Les
      1. Down site implementation, i.e. auto discover and report down sites : Adnan, Les
      2. Step changes (Brian Tierney LBL is interested in predictions): Waqar, Adnan
        1. Akbar is looking at this

          1. Speed up plateau by doing in 'C' [Drop 9/1/06]: Akbar

      3. Add check for nodes in NODENODETAILS that are not in pingtable.pl world to world [Done 8/15/06]: Akbar

      4. Clean up data: Jerrod, Les
      5. Review beacons: Jerrod

      6. Validate the IP addresses used by the monitoring sites against those in Guthrie [Done 8/12/06]: Akbar

      7. Clean up monitoring sites
        1. Email sent to ICTP 6/5/06 (responded 6/7/06, awaits director return, they agree 7/20/06, sent reminder 9/4/06): Les
      8. Convert offsite.nodes to a Guthrie database: Jerrod

        1. Convert Beacons to Guthrie and make sure they are identical [Done 8/10/06]: Jerrod

        2. Integrate with existing code
        3. Provide a web interface to view and edit Guthrie database entries : Jerrod
        4. Add contact names etc.
        5. Validate the lat/longs: Jerrod
        6. Enable entering the Group (Jerrod is working on 7/28/06): Jerrod
      9. Clean up sub-regions: Akbar

        1. Provide CGI script to enable adding countries to groups [Done 8/16/06]: Akbar

        2. Complete Mediterranean [Done 8/16/06]: Akbar, Les

        3. Fix up Africa [Done 8/21/06]: Akbar

      10. Look at anomalies for various metrics by region and country, define project: Akbar
      11. Make sure Maxim's monitoring hosts are current: Jerrod
    5. PingER Visualization, Shahryar
      1. Provide a visualroute capability: Shahryar
      2. Look at integration with TULIP: Shahryar, Faran
      3. Create a map with monitor and remote sites that will enable displaying of path characteristics RTT: Shahryar
      4. Display both lat and long in clickable list: Shahryar
      5. Add down site information on demand (test with ping & synack for given pair): Shahryar
      6. Fix names persisting outside map following mouseover: Shahryar
      7. Allow identification of nodes in top left hand corner (was done 8/3/06, but has come back 8/7/06, will fix when gets to SLAC): Shahryar

      8. Plot data seem to be missing (from SLAC site) since July 7 (reported 8/15/06) [Done 8/27/06 caused by bad pingtable data]: Shahryar

      9. Add minimum RTT and IQR to metrics (agreed 7/20/06, hope to have on 8/4/06): Shahryar
      10. Increase zoom level (agreed 7/20/06, increased by a couple of levels, looking at Google maps 8/7/06): Shahryar
      11. Add Beacon sites (need to install at SLAC): Shahryar
      12. Allow selection of all remote nodes (agreed 7/20/06): Shahryar
      13. Work on plots: Shahryar
      14. Allow user to specify tick time: Shahryar
      15. Plot multi metrics (e.g. RTT & loss, agreed 6/7/06): Shahryar
      16. Enable selecting multiple remote sites - Sharryar
      17. Use smokeping plots for variances and possibly uptime (awaits PDA) : Shahryar, Asma
      18. WAN Topology
      19. Extend to PDA, initial target HP iPac (in progress 6/6/06): Shahryar
      20. Add 3D capability (tryhttp://pinger2.niit.edu.pk/~sheryar/): Shahryar
    6. PingER & AMP Integration (AMP/NLANR has run out of funding, AMP will shut down 8/31/06): Abdullah, Adnan

      1. List of IP to be prepared and sent to Abdullah [Done for NIIT 6/6/06]: Les, Jerrod, Adnan

      2. Extend the addresses to add more source nodes: Abdullah

      3. Code to be sent by Abdullah to SLAC [Done 6/6/06, it converts data from a single AMP day]: Abdullah

      4. Install & integrate code at SLAC: Abdullah

      5. Send code to SLAC [Done 8/20/06]: Abdullah

      6. Review code and make suggestions [Done 8/21/06]: Les

      7. Improve code, make production quality, automate getting data (requested with suggestions 8/20/06, makimg progress hope to be done in 3-4 days 8/31/06): Abdullah
      8. Install at SLAC & integrate: Abdullah
      9. Get account: Abdullah

      10. Send configuration file (it is installed at /afs/slac/g/www/www-iepm/pinger/ampnodes.txt) [Done 8/22/06]: Abdullah

      11. Figure out how to automate getting data now it needs userid and password [Done 8/20/06 looks like can do with wget]: Abdullah

        1. Send email to AMP to see if there is a good way to get data (agreed 8/17/06) [Done 8/18/06]: Abdullah

      12. Integrate with pingtable.pl: Abdullah, Adnan
        1. Create AMP file directory: Jerrod
        2. Set up daily crontab: Jerrod
        3. Set up analyze-hourly etc: Jerrod
        4. Add src & dst amp nodes to Guthrie: Jerrod

        5. Choose AMP sites and manually populate directory for current day [Done 5/11/06]: Aziz, Jerrod

        6. Think about retrieving older data (find out how long ago there is data for): Jerrod, Adnan
        7. AMP and Pinger data comparison: Abdullah
    7. Smokeping & PingER Integration: Asma (Asma plans to move on in coming months, need to complete current tasks 8/3/106), Yee
      1. Look at how to input PingER data to Smokeping display module (sounds like we have to convert from PingER to RRD format, can this be done in realtime or do we need an RRD database too): Asma
      2. Look at how to to take recent smokeping data and convert to PingER format (in progress 7/19/06, needs pingtable.pl to get all 12 metrics)
      3. Install on pinger2.niit.edu.pk (authentication issue fixed having probalems with adding one day 8/31/06) - Asma
      4. Change from 90 to 120 days - Asma
      5. Integrate with Pingtable.pl (Jerrod will send pingtable.pl links 8/31/06): Asma
      6. Look at adding visualization of uptime: Asma
      7. Look at how to dynamically change the config file for Smokeping to accomodate the step interval (maybe time consuming, may affect all users and be cumbersome): Asma
      8. Look at using Smokeping to determine the quality of home links in Pakistan: Asma
      9. Will use as a top level access to PIngER and also as links from Pingtable to display last 45-90 days using smokeping: Asma, Yee
    8. Provide real-time display of PingER data:
      1. Export data to MonALISA from SLAC monitoring site:
      2. Figure how to deal with non regular (non exact 30 min data)
      3. Do for SLAC
        1. Decide on how to extend to other monitoring sites, try with NIIT: Akbar, Umer
        2. Hassan is looking into the code sent by Fawad (next step is to get ML web services client to get data): Akbar, Hassan
    9. Monitoring Hosts:
      1. http://sfsmds2.vsnl.net/cgi-bin/ping_data.pl has latest version of ping_data.pl, but no data is returned (sent email 9/8/06): Jerrod
      2. monitor.niit.edu.pk down since April 28th (email sent to Azher 7/13/06, 7/14/06, Azher responds 7/18/06, will install a new host, still no data 7/20/06, still no data 7/28/06, email sent to Azher 7/28/06): Azher, Jerrod
      3. Get pinger2.niit.edu.pk to insert the name (and not 203.99.50.206) for the monitoring host into the table of read data (probably needs a rebuild of PingER2, sent email to Azher he replies will do on 8/16/06, no data since Aug 18, reported 8/30/06, Azher expects to look at and undersand in 2-3 days 8/31/06): Azher
      4. pinger.pern.edu.pk no data since Aug 22nd (reported 8/30/06, down since Sep 6 email to Azher 9/8/06): Azher, Jerrod
      5. pinger.comsats no data since Sep 5, sent email 9/8/06): Azher, Jerrod
      6. Sent Azher mail on 9/1/06 and 9/6/06 regarding the need to update his Pinger configuration to refer to the node 202.83.166.42 by DNS name and not IPaddress. This difference is causing warnings to show up in the execution of runall.pl (reported on 9/6/06 - Jerrodw):                                                                                                                                             Warning: found 202.83.166.42 as monitoring host for monitoring site at 202.83.160.42:202.83.166.42 202.83.166.42 ping.slac.stanford.edu 134.79.18.21 100 1156982718 10 10 302.380 311.416 326.278 1 2 3 4 5 6
          There may be an alias between 202.83.166.42 and 202.83.160.42,
          or you may need to inform the contact at 202.83.160.42 that the configuration file is incorrect
          (e.g. the monitoring site is using its IP address rather than its name)
      7. NTC hosts, they agreed to install PingER2 at their 3 backbone sites
        (sent email to NTC 3/7/06 (Aziz sent email to Wajid Ali for Lahore & Karachi 3/14/06, they have asked for a physical host from NIIT, Azher will provide 4/20/06, NTC Lahore is 2nd in Azher's priority, NTC Karachi is 3rd 7/28/06, discussed with Arshad will provide host in a month 8/17/06, Azher needs to coordinate sending machine to remote location 8/31/06): Azher
    10. PingER2 improvements
      1. Provide multiple sources for beacons file: Warren
  2. Triangulate position of an unknown host (see http://www.slac.stanford.edu/comp/net/wan-mon/tulip,Faran back from ITCN 8/21/06): Faran
    1. Deployment: Faran
      1. Provide a program API: Faran, Shahryar
      2. Does not respond to Locate (with www.cern.ch), Status Window says "Loading..." (sent email 9/13/06): Faran

      3. Version at SLAC is not working (reported 8/9/06) [Done 8/21/06]: Faran

    2. Consider the security aspects of TULIP: Faran
      1. Add contact info in landmark ping packets for Linux landmarks using the -p (pad option): Les
      2. Maybe develop as servlet or CGI script: Faran
    3. Visualization
      1. Make tables sortable (working on 5/25/06, re-agreed 8/3/06, re-requested 8/21/06, done for the RTTs 8/22/06, can we do with buttons for all columns): Faran
      2. Can we add AS's to routes: Yee, Faran
      3. Integrate with Google maps: Faran

      4. Position landmarks properly on map (DL, is incorrectly located, reminded 6/19/06, reported still not right 6/22/06, Boston is incorrect, email sent 7/26/06, Les sent lat/longs 7/26/06) [Fixed 7/26/06]: Faran

      5. Provide diagnosis for non locateable, non-parseable, non responding paths (so far have NAN for non responsive landmark 6/6/06, reminded 6/19/06) [Done 8/2/06]: Faran

      6. When zooming maintain size of dots: Faran
      7. Do not truncate the information in the Ping Raw Data window: Faran
      8. Make Counttry/State names be consistent (requested 8/28/06): Faran
      9. Improve the diagnostic/debugging information for failures (requested 8/21/06): Faran

      10. Add estimates from other sources, e.g. DNS LOC records, GeoLoc [Done GeoLoc is in Log, 8/3/06]: Faran

    4. Landmarks
      1. Get more Traceroute Sites to TULIP:
        1. See if AMP interested (sent email 3/7/06, yes they appear to be, we have identified suitable landmarks, they will start with NZ 3/9/06, sent reminder 3/21/06 & 4/3/06, 4/18/06, 5/13/06, 6/21/06, more email, talking to kc 8/16/06): Les

        2. Try Gloriad/Russia (email sent 4/21/06, got response, sent more info 5/6/06, sent reminder 5/24/06, sent reminder 7/26/06, have contact for RBnet 8/2/06) [Done 8/16/06]: Les

        3. Send email to Alaska (5/7/06): Les
        4. Send email to Singapore (got response 5/3/06, sent reminder 5/24/06, got response 5/25/06, sent email 7/26/06): Les
        5. Send email to Malaysia (5/7/06): Les
        6. Sent email Latin America (5/7/06): Les
        7. Sent email for China contacts (7/26/06): Les

        8. Add landmarks at Mumbai, http://59.165.131.15/cgi-bin/traceroute.pl and http://pinger.pern.edu.pk/cgi-bin/traceroute.pl [Done 8/21/06]: Faran

        9. COMTEC, Japan(Faran send email reminder 5/25/06, agreed to send another reminder 6/7/06): Faran, Farooq
      2. Add NetDiGi route servers (asked for 8/30/06): Faran
      3. Add Abilene Looking Glasses (asked for 8/30/06): Faran
      4. Use PlanetLab for landmarks
        1. Join PlanetLab: Faran
        2. Provide a daemon for the ping server: Faran
        3. Provide a client for the TULIP: Faran
      5. Method for updating the landmarks: Faran
        1. Faran has implemented an edit function but unable to make it work at SLAC (5/25/06, demonstrated to Faran 6/21/06, Akbar sent email that modification is not working 7/25/06, Faran said retry but still not working 7/26/06): Faran

        2. Get add feature to work [Done 5/29/06]: Faran, Les

      6. Landmarks Fix/Add:
        1. pinger.ntc.net.pk denies accesss (email to Azher & Faran 7/25/06, Azher responded that he is checking 7/27/06, email sent again 8/21/06, Azher knows about will send person to repair 8/22/06): Faran/Azher
        2. Add UMich (requested 9/4/06) - Faran

        3. bnl.gov needs to be changed to bnl.org [Done 8/22/06]: Faran

        4. Checking wget http://sfsmds2.vsnl.net/cgi-bin/traceroute.pl?function=ping&target=www.cern.ch&contact=cottrell@slac.stanford.edu&id=TULIP works with wget from www.slac.stanford.edu, but not from TULIP and not from atreides.slac.stanford.edu windows (Firefox) browser. Turns out pings from atreides and iepm-bw are blocked but not from pinger. (sent email to Mumbai explaining problem and suggesting using a non-standard port 8/21/06) [Done 8/28/06 opened port 80 to all]: Les-

    5. Apply to PingER node database to check the lat/longs are roughly correct (tried with 300-400 sites but cannnot get results due to fragile Internet at NIIT, will interact with Akbar to make tests at SLAC 8/31/06, exchanged emails on list of PingER hosts to locate 9/3/06): Faran
    6. Analysis

      1. Develop new (alpha free) method to estimate location (in progress, expect to resolve in 2-3 days 7/20/06) [Done did not work out 8/3/06]: Faran

      2. Literature search on triangulation, multi-lateration: Faran

      3. Document host names/addresses with multiple locations (e.g. root name servers) [Done 9/9/06]: Les

      4. Compare with DNS LOC records, plus availability of DNS LOCs: Faran
      5. Compare with IP2Location and/or Netgeo records (from http://netgeo.caida.org/aslatlong.txt): Faran
      6. Document Faran's algorithm, try new algorithm with optimized alpha that does not require destination to be inside triangle: Faran, Umar
      7. Compare Faran's algorithm with Bamba's, Octant & simple triangulation
        1. Contact Bamba to find out if http://planetlab-01.ipv6.lip6.fr:10000/cbg.php works for any sites (email send 4/21/06): Aziz
        2. Confidence estimates
        3. Applied to PingER hosts
        4. Outside triangles, area coverage, need map with accuracy
    7. Analysis of Accuracy and applicability
      1. Accuracy vs known locations
      2. Effect of removing, adding landmarks
      3. Comparison with other sets of landmarks & methods (can't do Bamba since it is down), Octant, NetGeo for accuracy coverage etc.
      4. Provide bubble maps of accuracy: Faran
      5. What area/population is covered by TULIP
      6. Document anomalies  (e.g. root-servers): Faran
  3. NETRAT, SLAC contact: Adnan, Les
    1. Evaluate PCA multivariate method of Crovella: Waqar, Adnan,
      1. Compare PCA results with Holt-Winters/plateau: Adnan
        1. Fetch Data From SLAC, BNL, CALTECH (Pathchirp, Thrulay, Ping): Adnan
        2. Fetch data from CERN
        3. Apply PCA on same data and to get potential alerts: Adnan
        4. Comparison of Run-period results and PCA results: Adnan
        5. Improve event detector: Adnan
        6. Docment findings on PCA: Adnan
    2. Forecasting comparison of Active vs. passive data: Fareena, Adnan
      1. Look at eview implementation of ARIMA (QU has copy): Fareena
        1. Extend to more sites/metrics, see need for Integration & higher orders: Farena, Adnan
        2. Try with longer time base: Fareena
        3. Get time estimates : Fareena

      2. Look at JMSL (license $3500 for developer, $2500 for user) [Done 8/17/06]

      3. Detailed Graphs description: Fareena
      4. Regularization
      5. Look at moodss (http://jfontain.free.fr/statistics.htm): Fareena
      6. Send results/documentation to SLAC (will send 8/31/06): Fareena
      7. Automate procedures
        1. Define project
        2. Put together presentation for workshop (for Sep 2006): Fareena
        3. Gather team
    3. Automatic event isolation: Adnan, Yee
      1. Background/introductory studies: Ali
      2. Canonical Data set: Adnan
        1. Fetch data from monitoring nodes (Pathchirp, Thrulay, Ping, Traceroute): Adnan, Ali
        2. Study/Analyze Data: Adnan, Ali
        3. Run Run-period: Adnan
      3. ALERT table changes
        1. New table design: Adnan
        2. Script to populate the table: Adnan
        3. database connectivity problem: Jerrod
        4. Modify other scripts accordingly: Connie
  4. IEPM, SLAC contact Connie
    1. Install OWAMP at NIIT, get NTP working well (NTP working according to Azher 4/20/06): Connie, Azher
    2. IEPM-BW Web-services:
      1. Waiting for a new student: Fawad, Umar
  5. Make traceanal into stand alone package: Connie, Umar find someone new to take over
    1. Add in new topology: Connie
    2. See if can be modularized: Yee
    3. Integrate with AMP (need pingtable form like interface to allow selection of sce/dst sites and date).

    4. Define project (check how to get data from AMP) [Done 7/25/06]: Les, Aziz

    5. Apply to PerfSONAR data
  6. Pathneck (see http://maggie.niit.edu.pk/website/projects/Topological_Analysis.html) Yee is the lead at the SLAC end: Asif
    1. Try and get pathneck working from NIIT, study debug output, work with Ningning <hnn@cs.cmu.edu> (awaits site upgrade): Asif
    2. Develop and install pathneck reverse server (would like to start at NIIT just to assure code works work with -y option): Les, Asif
      1. Modularize security of traceroute.pl: Asif, Yee
      2. Add in PerfSonar: Asif, Yee
    3. Evaluate and understand how pathneck needs to be configured, the failure modes, the areas of applicability etc. using iepm-resp.slac.stanford.edu: Asif
      1. Provide web site documenting failure modes: Asif
    4. Develop a vizualization
      1. Topology (currently with Touchgraph), color links, mouseover ID router, provide snapshot
      2. Click on link/router to get graph
      3. Incorporate location (e.g. via TULIP)
    5. Investigate integration with traceanal and graphviz: Asif
    6. Integrate topology map with pathneck: Asif
    7. Look at using PerfSONAR router information (do not agree well, need to get more statistics, to enable multiple measures need API to get PerfSONAR data, look perfsonar UI): Asif
    8. Try on Ultralight (problems if < 4 hops): Asif
    9. Problem of 0 results (reported to author but not much happening 4/20/06): Asif
    10. Add position and AS information to router database: Asif
  7. Communications etc.
    1. Shahryar & Asif have visas will arrive Thursday 14th September around noon: Shahryar
    2. Arshad will arrive evening Sep 11, spend 12-14 in Bay Area, may visit UCB on 14th, then fly to LA: Arshad
      1. Arrange meetings, Jonathan, Neil, Mahesh, Aziz, Bebo: Adnan, Les
    3. Paper redo for CHEP06 (submitted by Jerrod 5/24/06, need to get registered as SLAC PUB) - Aziz
    4. CHEP06 Netflow paper (Umar contacted Fawad 5/17/06): Fawad, Les, Akbar
    5. Paper for HONET (Fareena's paper has been accepted, needs travel money, sent comments to Fareena on paper 9/3/06): Fareena

    6. Paper for AACC Kathmandu [Done 8/31/06]: Les, Akbar, Umar

    7. Improve reliabilty of mail list server: Azher
      1. Install new UPS (due 7/23/06): Azher
      2. Use maggie-niit@googlegroups.com as alternative 8/3/06: Azher
    8. Improve NIIT connectivity: Azher
      1. Move from NTC to PTCL to remove finger pointing, and upgrade from < 1.5Mbps DSL to 2Mbits/s using fiber pedestal 3 blocks from NIIT with Cu from NIIT to pedestal (recommended, expect to start in a months and for instability during cut-over 7/20/06, have prepared all the docs for submission to PTCL, and on tuesday we will be submiiting them to ptcl for further necessary actions 9/4/06): Azher
    9. Projects (see http://www-iepm.slac.stanford.edu/pinger/projects/)
      1. Two possible new students are: Mauman Shah and Qasim Ali
    10. Proposals
      1. Submitted NTC proposal (sent for review by PTCL 8/3/06 ): Arshad
      2. Pakistan US Cooperative Grants Program 2006 (submitted awaiting review, 8/3/06)

        1. Need to focus on aiding the people & environment (e.g. Digital Divide, distance learning) [Done 8/3/06]

        2. Review old proposal (Les gave copy to Adnan 5/11/06) [Done 7/17/06]: Adnan

        3. Prepare proposal [Done 7/17/06]: Les, Umar, Yee

        4. Get signed by SLAC, Stanford (expect 7/27/06) [Done 8/3/06]: Les

      3. HEC solicitation for International collaborations, due mid-Sept, Farooq PI

        1. Create draft [Done 9/3/06]: Umar

        2. Review draft: Les
  8. Wiki
    1. Get accounts/passwords for NIIT students - Yee
    2. Create space for NIIT student reports and populate with initial reports - Yee, Les
  9. Maggie-NS Site: Umar
    1. Need to be updated with new students photos and profile.
      1. Adding Student Project Description.
      2. Some Pictures Left and will be added soon
      3. Post the weekly presentation on the maggie-ns web site.
  • No labels