Maggie-NS Tasklist
Last Updated : August 21, 2006

NEXT MEETING: 9:00 AM California, USA Time, Thursday September 31st, 2006

Team members: Abdullah, Arshad*, Asif, Asma*, Azher*, Faran*, Fareena*, Rabail*, Shahryar*, Umar**

Adnan, Akbar*, Connie*, Jerrod*, Les*, Yee**

Mailing lists: maggie-niit@ns.niit.edu.pk; backup: maggie-niit@googlegroups.com

Awaits something, also provides start of wait date
Done or drop, remove < 2 months are task done
: Person doing the work
Bold face: task in progress, or for discussion at meeting
Recent progress/discussion. Need student to be assigned. Critical.

Action Items

  1. PingER contacts: Akbar Mehdi, Umar Khan
    1. Remove regions East Europe [Done 7/13/06]: Les, Jerrod
    2. Provide tools to analyze and create plots of PingER data: Akbar, Les
      1. Tie into webservices/SQL access to data to make a much more flexible distributed way of selecting analyzing and displaying data
      2. Regional plots for executives  using JFreeChart: Akbar
        1. Create a web page which displays different pinger executive graphs. (has a prototype 4/18/06, need to test): Akbar
          1. Enter should be same as Apply changes in Line patterns, also allow selection of colors: Akbar
          2. Provide cronjob capability (need to understand what is wanted): Akbar
          3. Retain knowledge of most recent directory in Save As and Directly Open File [Done 5/15/06]: Akbar
      3. Provide plots of yearly loss, RTT and throughput (i.e. one point/year, need to get more accurate loss measures and thus derived throughput for excellent paths): Les, Aziz
    3. PingER web services: ?
    4. Pinger Management project: Jerrod, Les
      1. Down site implementation, i.e. auto discover and report down sites : Adnan, Les
      2. Step changes (Brian Tierney LBL is interested in predictions): Waqar, Adnan
        1. Akbar is looking at this
          1. Speed up plateau by doing in 'C': Akbar
      3. Add check for nodes in NODENODETAILS that are not in pingtable.pl world to world [Done 8/15/06]: Akbar
      4. Clean up data: Jerrod, Les
      5. Review beacons: Jerrod
      6. Validate the IP addresses used by the monitoring sites against those in Guthrie [Done 8/12/06]: Akbar
      7. Clean up monitoring sites
        1. Email sent to ICTP 6/5/06 (responded 6/7/06, awaits director return, they agree 7/20/06): Les
      8. Convert offsite.nodes to a Guthrie database: Jerrod
        1. Convert Beacons to Guthrie and make sure they are identical [Done 8/10/06]: Jerrod
        2. Integrate with existing code
        3. Provide a web interface to view and edit Guthrie database entries : Jerrod
        4. Add contact names etc.
        5. Validate the lat/longs: Jerrod
        6. Enable entering the Group (Jerrod is working on 7/28/06): Jerrod
      9. Clean up sub-regions: Akbar
        1. Provide CGI script to enable adding countries to groups [Done 8/16/06]: Akbar
        2. Complete Mediterranean [Done 8/16/06]: Akbar, Les
        3. Fix up Africa [Done 8/21/06]: Akbar
      10. Look at anomalies for various metrics by region and country, define project: Akbar
      11. Make sure Maxim's monitoring hosts are current: Jerrod
    5. PingER Visualization, Shahryar
      1. Provide a visualroute capability: Shahryar
      2. Look at integration with TULIP: Shahryar, Faran
      3. Create a map with monitor and remote sites that will enable displaying of path characteristics RTT: Shahryar
      4. Display both lat and long in clickable list: Shahryar
      5. Add down site information on demand (test with ping & synack for given pair): Shahryar
      6. Fix names persisting outside map following mouseover: Shahryar
      7. Allow identification of nodes in top left hand corner (was done 8/3/06, but has come back 8/7/06, will fix when gets to SLAC): Shahryar
      8. Plot data seem to be missing (from SLAC site) since July 7 (reported 8/15/06) [Done 8/27/06 caused by bad pingtable data]: Shahryar
      9. Add minimum RTT and IQR to metrics (agreed 7/20/06, hope to have on 8/4/06): Shahryar
      10. Increase zoom level (agreed 7/20/06, increased by a couple of levels, looking at Google maps 8/7/06): Shahryar
      11. Add Beacon sites (need to install at SLAC): Shahryar
      12. Allow selection of all remote nodes (agreed 7/20/06): Shahryar
      13. Work on plots: Shahryar
      14. Allow user to specify tick time: Shahryar
      15. Plot multi metrics (e.g. RTT & loss, agreed 6/7/06): Shahryar
      16. Enable selecting multiple remote sites - Sharryar
      17. Use smokeping plots for variances and possibly uptime (awaits PDA) : Shahryar, Asma
      18. WAN Topology
      19. Extend to PDA, initial target HP iPac (in progress 6/6/06): Shahryar
      20. Add 3D capability (tryhttp://pinger2.niit.edu.pk/~sheryar/): Shahryar
    6. PingER & AMP Integration (AMP/NLANR has run out of funding, AMP will shut down 8/31/06): Abdullah, Adnan
      1. List of IP to be prepared and sent to Abdullah [Done for NIIT 6/6/06]: Les, Jerrod, Adnan
      2. Extend the addresses to add more source nodes: Abdullah
      3. Code to be sent by Abdullah to SLAC [Done 6/6/06, it converts data from a single AMP day]: Abdullah
      4. Install & integrate code at SLAC: Abdullah
      5. Send code to SLAC [Done 8/20/06]: Abdullah
      6. Review code and make suggestions [Done 8/21/06]: Les
      7. Improve code, make production quality, automate getting data (requested with suggestions 8/20/06, makimg progress hope to be done in 3-4 days 8/31/06): Abdullah
      8. Install at SLAC & integrate: Abdullah
      9. Get account: Abdullah
      10. Send configuration file (it is installed at /afs/slac/g/www/www-iepm/pinger/ampnodes.txt) [Done 8/22/06]: Abdullah
      11. Figure out how to automate getting data now it needs userid and password [Done 8/20/06 looks like can do with wget]: Abdullah
        1. Send email to AMP to see if there is a good way to get data (agreed 8/17/06) [Done 8/18/06]: Abdullah
      12. Integrate with pingtable.pl: Abdullah, Adnan
        1. Create AMP file directory: Jerrod
        2. Set up daily crontab: Jerrod
        3. Set up analyze-hourly etc: Jerrod
        4. Add src & dst amp nodes to Guthrie: Jerrod
        5. Choose AMP sites and manually populate directory for current day [Done 5/11/06]: Aziz, Jerrod
        6. Think about retrieving older data (find out how long ago there is data for): Jerrod, Adnan
        7. AMP and Pinger data comparison: Abdullah
    7. Smokeping & PingER Integration: Asma (Asma plans to move on in coming months, need to complete current tasks 8/3/106), Yee
      1. Look at how to input PingER data to Smokeping display module (sounds like we have to convert from PingER to RRD format, can this be done in realtime or do we need an RRD database too): Asma
      2. Look at how to to take recent smokeping data and convert to PingER format (in progress 7/19/06, needs pingtable.pl to get all 12 metrics)
      3. Install on pinger2.niit.edu.pk (authentication issue fixed having probalems with adding one day 8/31/06) - Asma
      1. Change from 90 to 120 days - Asma
      1. Integrate with Pingtable.pl (Jerrod will send pingtable.pl links 8/31/06): Asma
      1. Look at adding visualization of uptime: Asma
      1. Look at how to dynamically change the config file for Smokeping to accomodate the step interval (maybe time consuming, may affect all users and be cumbersome): Asma
      2. Look at using Smokeping to determine the quality of home links in Pakistan: Asma
      3. Will use as a top level access to PIngER and also as links from Pingtable to display last 45-90 days using smokeping: Asma, Yee
    2. Provide real-time display of PingER data:
      1. Export data to MonALISA from SLAC monitoring site:
      2. Figure how to deal with non regular (non exact 30 min data)
      3. Do for SLAC
        1. Decide on how to extend to other monitoring sites, try with NIIT: Akbar, Umer
        2. Hassan is looking into the code sent by Fawad (next step is to get ML web services client to get data): Akbar, Hassan
    3. Monitoring Hosts:
      1. Install PingER at CDAC Mumbai (Binu Abraham, PingER2 installed but neither ping_data.pl nor traceroute.pl can be found, Aziz will send email 3/22/06, Binu keeps responding that will do it but does not, Aziz will prompt with cc to supervisor 4/18/06, Jerrod will follow up 7/20/06) [Done 8/15/06]: Jerrod, Les
      2. Install PingER2 at VSNL Mumbai aka http://59.165.131.15/cgi-bin/ping_data.pl (sent email to Sunil Abrahim 3/14/06, Sunil sent email to someone else, seems to be a problem with the measurements 3/21/06, Aziz will check status 4/18/06, Jerrod will follow up with email to bs.gupta@vsnl.co.in 7/20/06, ping_data.pl responds but no remote nodes also need to get in DNS, sent email 8/16/06) [Done 8/21/06]: Jerrod
      3. http://sfsmds2.vsnl.net/cgi-bin/ping_data.pl has no remote hosts and is using an out of date version of ping_data.pl (notified  8/22/06 & 8/28/06, Mantosh Kharkar awaits Jerrod's retuurn 8/30/06) - Jerrod
      4. monitor.niit.edu.pk down since April 28th (email sent to Azher 7/13/06, 7/14/06, Azher responds 7/18/06, will install a new host, still no data 7/20/06, still no data 7/28/06, email sent to Azher 7/28/06): Azher, Jerrod
      5. Get pinger2.niit.edu.pk to insert the name (and not 203.99.50.206) for the monitoring host into the table of read data (probably needs a rebuild of PingER2, sent email to Azher he replies will do on 8/16/06, no data since Aug 18, reported 8/30/06, Azher expects to look at and undersand in 2-3 days 8/31/06): Azher
      6. pinger.pern.edu.pk no data since Aug 22nd (reported 8/30/06): Azher
      7. PERN/HEC Monitoring host(Islamabad)-Email sent to Asim Raza 2/21/06 we await hearing from him (emailed again 3/14/06, Asim responded, needed help, we await URL of how to gather data 4/13/06, Azher will contact 4/21/06, 7/8/06 pings blocked have requested unblock, Azher will install in Islamabad once network probs are fixed, i.e. it is 1st priority for Azher to be done by 7/27/06 7/20/06, not done & email sent 7/28/06, Azher has access to the system hopes to install pinger2. He has completed the installation and it is taking data http://pinger.pern.edu.pk/cgi-bin/ping-data.pl 8/16/06) [Done 8/16/06]: Azher
      8. NTC hosts, they agreed to install PingER2 at their 3 backbone sites
        (sent email to NTC 3/7/06 (Aziz sent email to Wajid Ali for Lahore & Karachi 3/14/06, they have asked for a physical host from NIIT, Azher will provide 4/20/06, NTC Lahore is 2nd in Azher's priority, NTC Karachi is 3rd 7/28/06, discussed with Arshad will provide host in a month 8/17/06, Azher needs to coordinate sending machine to remote location 8/31/06): Azher
      9. Improve list of hosts monitored from pinger2.niit.edu.pk (remove aiou and monalisa) [Done 8/27/06]: Azher
    4. PingER2 improvements
      1. Provide multiple sources for beacons file: Warren
  2. Triangulate position of an unknown host (see http://www.slac.stanford.edu/comp/net/wan-mon/tulip,Faran back from ITCN 8/21/06): Faran
    1. Deployment: Faran
      1. Provide a program API: Faran, Shahryar
      2. Version at SLAC is not working (reported 8/9/06) [Done 8/21/06]: Faran
    2. Consider the security aspects of TULIP: Faran
      1. Add contact info in landmark ping packets for Linux landmarks using the -p (pad option): Les
      2. Maybe develop as servlet or CGI script: Faran
    3. Visualization
      1. Make tables sortable (working on 5/25/06, re-agreed 8/3/06, re-requested 8/21/06, done for the RTTs 8/22/06, can we do with buttons for all columns): Faran
      2. Can we add AS's to routes: Yee, Faran
      3. Integrate with Google maps: Faran
      4. Position landmarks properly on map (DL, is incorrectly located, reminded 6/19/06, reported still not right 6/22/06, Boston is incorrect, email sent 7/26/06, Les sent lat/longs 7/26/06) [Fixed 7/26/06]: Faran
      5. Provide diagnosis for non locateable, non-parseable, non responding paths (so far have NAN for non responsive landmark 6/6/06, reminded 6/19/06) [Done 8/2/06]: Faran
      6. When zooming maintain size of dots: Faran
      7. Do not truncate the information in the Ping Raw Data window: Faran
      8. Make Counttry/State names be consistent (requested 8/28/06): Faran
      9. Improve the diagnostic/debugging information for failures (requested 8/21/06): Faran
      10. Add estimates from other sources, e.g. DNS LOC records, GeoLoc [Done GeoLoc is in Log, 8/3/06]: Faran
    4. Landmarks
      1. Get more Traceroute Sites to TULIP:
        1. See if AMP interested (sent email 3/7/06, yes they appear to be, we have identified suitable landmarks, they will start with NZ 3/9/06, sent reminder 3/21/06 & 4/3/06, 4/18/06, 5/13/06, 6/21/06, more email, talking to kc 8/16/06): Les
        2. Try Gloriad/Russia (email sent 4/21/06, got response, sent more info 5/6/06, sent reminder 5/24/06, sent reminder 7/26/06, have contact for RBnet 8/2/06) [Done 8/16/06]: Les
        3. Send email to Alaska (5/7/06): Les
        4. Send email to Singapore (got response 5/3/06, sent reminder 5/24/06, got response 5/25/06, sent email 7/26/06): Les
        5. Send email to Malaysia (5/7/06): Les
        6. Sent email Latin America (5/7/06): Les
        7. Sent email for China contacts (7/26/06): Les
        8. Add landmarks at Mumbai, http://59.165.131.15/cgi-bin/traceroute.pl and http://pinger.pern.edu.pk/cgi-bin/traceroute.pl [Done 8/21/06]: Faran
        9. COMTEC, Japan(Faran send email reminder 5/25/06, agreed to send another reminder 6/7/06): Faran, Farooq
      2. Add NetDiGi route servers (asked for 8/30/06): Faran
      3. Add Abilene Looking Glasses (asked for 8/30/06): Faran
      4. Use PlanetLab for landmarks
        1. Join PlanetLab: Faran
        2. Provide a daemon for the ping server: Faran
        3. Provide a client for the TULIP: Faran
      5. Method for updating the landmarks: Faran
        1. Faran has implemented an edit function but unable to make it work at SLAC (5/25/06, demonstrated to Faran 6/21/06, Akbar sent email that modification is not working 7/25/06, Faran said retry but still not working 7/26/06): Faran
        2. Get add feature to work [Done 5/29/06]: Faran, Les
      6. Landmarks:
        1. pinger.ntc.net.pk denies accesss (email to Azher & Faran 7/25/06, Azher responded that he is checking 7/27/06, email sent again 8/21/06, Azher knows about will send person to repair 8/22/06): Faran/Azher
        2. 202.83.166.182/cgi-bin/mati (Akbar sent email to Faran & Azher 7/25/06, sent email to Azher, is it still in use 8/21/06) [Done 8/21/06, decide to remove the landmarks since have others at NIIT]: Faran/Azher
        3. 202.83.160.42 aka pinger.ntc.net.pk (Akbar sent email to Azher & Faran 7/25/06) [Fixed 8/21/06]: Faran/Azher
        4. Fix or understand why the Looking Glass sites are not working (7/26/06) [Done 8/16/06]: Faran, Les, Akbar
        5. bnl.gov needs to be changed to bnl.org [Done 8/22/06]: Faran
        6. Checking wget http://sfsmds2.vsnl.net/cgi-bin/traceroute.pl?function=ping&target=www.cern.ch&contact=cottrell@slac.stanford.edu&id=TULIP works with wget from www.slac.stanford.edu, but not from TULIP and not from atreides.slac.stanford.edu windows (Firefox) browser. Turns out pings from atreides and iepm-bw are blocked but not from pinger. (sent email to Mumbai explaining problem and suggesting using a non-standard port 8/21/06) [Done 8/28/06 opened port 80 to all]: Les-
    5. Apply to PingER node database to check the lat/longs are roughly correct (tried with 300-400 sites but cannnot get results due to fragile Internet at NIIT, will interact with Akbar to make tests at SLAC 8/31/06): Faran
    6. Analysis
      1. Develop new (alpha free) method to estimate location (in progress, expect to resolve in 2-3 days 7/20/06) [Done did not work out 8/3/06]: Faran
      2. Literature search on triangulation, multi-lateration: Faran
      3. Compare with DNS LOC records, plus availability of DNS LOCs: Faran
      4. Compare with IP2Location and/or Netgeo records (from http://netgeo.caida.org/aslatlong.txt): Faran
      5. Document Faran's algorithm, try new algorithm with optimized alpha that does not require destination to be inside triangle: Faran, Umar
      6. Compare Faran's algorithm with Bamba's, Octant & simple triangulation
        1. Contact Bamba to find out if http://planetlab-01.ipv6.lip6.fr:10000/cbg.php works for any sites (email send 4/21/06): Aziz
        2. Confidence estimates
        3. Applied to PingER hosts
        4. Outside triangles, area coverage, need map with accuracy
    7. Analysis of Accuracy and applicability
      1. Accuracy vs known locations
      2. Effect of removing, adding landmarks
      3. Comparison with other sets of landmarks & methods (can't do Bamba since it is down), Octant, NetGeo for accuracy coverage etc.
      4. Provide bubble maps of accuracy: Faran
      5. What area/population is covered by TULIP
  3. NETRAT, SLAC contact: Adnan, Les
    1. Evaluate PCA multivariate method of Crovella: Waqar, Adnan,
      1. Compare PCA results with Holt-Winters/plateau: Adnan
        1. Fetch Data From SLAC, BNL, CALTECH (Pathchirp, Thrulay, Ping): Adnan
        2. Fetch data from CERN
        3. Apply PCA on same data and to get potential alerts: Adnan
        4. Comparison of Run-period results and PCA results: Adnan
        5. Improve event detector: Adnan
        6. Docment findings on PCA: Adnan
    2. Forecasting comparison of Active vs. passive data: Fareena, Adnan
      1. Look at eview implementation of ARIMA (QU has copy): Fareena
        1. Extend to more sites/metrics, see need for Integration & higher orders: Farena, Adnan
        2. Try with longer time base: Fareena
        3. Get time estimates : Fareena
      2. Look at JMSL (license $3500 for developer, $2500 for user) [Done 8/17/06]
      3. Detailed Graphs description: Fareena
      4. Regularization
      5. Look at moodss (http://jfontain.free.fr/statistics.htm): Fareena
      6. Send results/documentation to SLAC (will send 8/31/06): Fareena
      1. Automate procedures
        1. Define project
        2. Put together presentation for workshop (for Sep 2006): Fareena
        3. Gather team
    2. Automatic event isolation: Adnan, Yee
      1. Background/introductory studies: Ali
      2. Canonical Data set: Adnan
        1. Fetch data from monitoring nodes (Pathchirp, Thrulay, Ping, Traceroute): Adnan, Ali
        2. Study/Analyze Data: Adnan, Ali
        3. Run Run-period: Adnan
      3. ALERT table changes
        1. New table design: Adnan
        2. Script to populate the table: Adnan
        3. database connectivity problem: Jerrod
        4. Modify other scripts accordingly: Connie
  2. IEPM, SLAC contact Connie
    1. Install OWAMP at NIIT, get NTP working well (NTP working according to Azher 4/20/06): Connie, Azher
    2. IEPM-BW Web-services:
      1. Waiting for a new student: Fawad, Umar
  3. Make traceanal into stand alone package: Connie, Umar find someone new to take over
    1. Add in new topology: Connie
    2. See if can be modularized: Yee
    3. Integrate with AMP (need pingtable form like interface to allow selection of sce/dst sites and date).
    4. Define project (check how to get data from AMP) Done 7/25/06: Les, Aziz
    5. Apply to PerfSONAR data
  4. Pathneck (see http://maggie.niit.edu.pk/website/projects/Topological_Analysis.html) Yee is the lead at the SLAC end: Asif
    1. Try and get pathneck working from NIIT, study debug output, work with Ningning <hnn@cs.cmu.edu> (awaits site upgrade): Asif
    2. Develop and install pathneck reverse server (would like to start at NIIT just to assure code works work with -y option): Les, Asif
      1. Modularize security of traceroute.pl: Asif, Yee
      2. Add in PerfSonar: Asif, Yee
    3. Evaluate and understand how pathneck needs to be configured, the failure modes, the areas of applicability etc. using iepm-resp.slac.stanford.edu: Asif
      1. Provide web site documenting failure modes: Asif
    4. Develop a vizualization
      1. Topology (currently with Touchgraph), color links, mouseover ID router, provide snapshot
      2. Click on link/router to get graph
      3. Incorporate location (e.g. via TULIP)
    5. Investigate integration with traceanal and graphviz: Asif
    6. Integrate topology map with pathneck: Asif
    7. Look at using PerfSONAR router information (do not agree well, need to get more statistics, to enable multiple measures need API to get PerfSONAR data, look perfsonar UI): Asif
    8. Try on Ultralight (problems if < 4 hops): Asif
    9. Problem of 0 results (reported to author but not much happening 4/20/06): Asif
    10. Add position and AS information to router database: Asif
  5. Communications etc.
    1. Shahryar & Asif have visas will arrive Thursday 14th September around noon: Shahryar
    2. Arshad will arrive evening Sep 11, spend 12-14 in Bay Area, may visit UCB on 14th, then fly to LA: Arshad
      1. Arrange meetings, Jonathan, Neil, Mahesh, Aziz, Bebo: Adnan, Les
    3. Paper redo for CHEP06 (submitted by Jerrod 5/24/06, need to get registered as SLAC PUB) - Aziz
    4. CHEP06 Netflow paper (Umar contacted Fawad 5/17/06): Fawad, Les, Akbar
    5. Paper for HONET (Fareena's paper has been accepted, needs travel money): Fareena
    6. Paper for AACC Kathmandu
    7. Improve reliabilty of mail list server: Azher
      1. Install new UPS (due 7/23/06): Azher
      2. Use maggie-niit@googlegroups.com as alternative 8/3/06: Azher
    8. Improve NIIT connectivity: Azher
      1. Move from NTC to PTCL to remove finger pointing, and upgrade from < 1.5Mbps DSL to 2Mbits/s using fiber pedestal 3 blocks from NIIT with Cu from NIIT to pedestal (recommended, expect to start in a months and for instability during cut-over 7/20/06): Azher
    9. Projects (see http://www-iepm.slac.stanford.edu/pinger/projects/)
      1. Two possible new students are: Mauman Shah and Qasim Ali
    10. Proposals
      1. Submitted NTC proposal (sent for review by PTCL 8/3/06 ): Arshad
      2. Pakistan US Cooperative Grants Program 2006 (submitted awaiting review, 8/3/06)
        1. Need to focus on aiding the people & environment (e.g. Digital Divide, distance learning) [Done 8/3/06]
        2. Review old proposal (Les gave copy to Adnan 5/11/06) [Done 7/17/06]: Adnan
        3. Prepare proposal [Done 7/17/06]: Les, Umar, Yee
        4. Get signed by SLAC, Stanford (expect 7/27/06) [Done 8/3/06]: Les
      3. HEC solicitation for International collaborations, due mid-Sept, Farooq PI
        1. Create draft (hopes to have done soon 8/31/06): Umar
  6. Wiki
    1. Get accounts/passwords for NIIT students - Yee
    2. Create space for NIIT student rports and populate with initial reports - Yee, Less
  7. Maggie-NS Site: Umar
    1. Need to be updated with new students photos and profile.
      1. Adding Student Project Description.
      2. Some Pictures Left and will be added soon
      3. Post the weekly presentation on the maggie-ns web site.
  • No labels