IEPM-NIIT Task List Current

PingER: Shahryar, Les

1. PingER web services: ?
   1. Extend ViPER to turn into front end for PingER, fix bugs and  extend to Real Time (see ViPER Extension Details and Mapping PingER Metrics): Nauman (has left), Shahryar
      1. Nauman has a real time version working with ping_data.pl. looks good.
      2. Fix bugs
         1. Still have points in to LH corner
      3. Make JNLP loadable.
      4. Add long term so replace old ViPER
   2. Ping-data.pl form only shows beacons.
      1. Need to point $sites_fn to pinger.xml instead of beacons.txt & check if the file is xml or flat (for backward compatibility): Shahryar
   3. Case Studies
      1. Case study for E. Asia: Umar, Farhan, Shahryrar
         1. Rough draft ready including comparison with other countries, needs more study to make presentable
   4. Monitoring Hosts:
      1. Contacts with Peru, Argentina, Mexico, Chile: Les
      2. Trying to re-energize African countries in association with IHY and Montpellier meetings
      3. Setup node in UNSW, AU: Umar

Triangulate position of an unknown host (see http://www.slac.stanford.edu/comp/net/wan-mon/tulip): Shahryar, Les

1. Visualization 
   1. When zooming maintain size of dots (awaits TULIP2)
2. Bugs/questions (await TULIP2):
   1. How do Sno correlate when there are landmarks from SLAC and PlanetLab. It appears Sno for PlanetLab in Ping Results is not same as Sno in PlanetLab landmarks? (currently it is by order of of appearance in Ping Results 10/2/07):
   2. The alpha optimization using PingER data seems to do nothing useful:
   3. Confidence Estimates table does not sort on table heading (what does it mean, what are units, will replace with IQR/Median 10/2/07):
   4. Location History needs headings or something to explain what the columns are:
   5. Change Ladnmarks to Landmarks on tab in Ping window (reported 9/21/07, will be done in future release 10/2/07):
   6. Sort of LG/SLAC ping results for target = www.cern.ch of sort of  Min RTT gives the top entry as 1 (SWITCH,Switzerland), the next is 0.820 (CERN, Geneva), i.e. it has mis-sorted (reported 10/8/07):
   7. PlanetLab ping results table does not sort by table heading (reported 10/8/07):
   8. The alpha test under Discovery Statistics needs to throw away useless values of RTT else plot scaling can be useless:
   9. The alpha optimization using PingER seems to do nothing useful:
   10. South Africa landmark only accepts IP addresses so need to accommodate:
   11. What is fixed vs Min RTT under DiscoveryStatistics (Faran will talk to Asif 10/2/07, not needed now):
   12. Sometimes fails, maybe because when tulip starts it downloads Site.txt from niit server. If that be the case the left side table on the main frame should be empty and status bar should say "Error downloading config. file."
3. Improvements/TULIP2 with Centralized Reflector
   1. Modify reflector.cgi to use the new sites.xml file, install at SLAC (done 1/12/08): Shahryar, Les
   2. Modify client to use new sites.xml file: Shahryar
   3. Redo NODEDETAILS database to clean up data , improve schema and generate the sites.xml file (done for landamrks only 11/30/07, awaits analyze hourly and ICFA year end report): Jared
      1. Clean up and add rest of PingER data: Jared
      2. Provide user interface
      3. Migrate from old to new version
   4. Allow alpha to change (lot of work, many changes, could improve, need analysis & understanding)
   5. Look at using PingER historical data to optimize alpha (agreed 4/12/07):
   6. Evaluate whether using traceroute data helps (Suggested by Umar & Les 2/26/07, could add 3 more RTTs to minimize from):
   7. Add traceroute location:
   8. Add results from GeoIP:
4. Landmarks
   1. Get more landmarks for TULIP, ideally need at least one landmark in all major developing countries that do not have direct connections to nearby countries.:
      1. Is IHEP in list?
5. Documentation
   1. Add information on the interpretive script sent to PlanetLab hosts: Shahryar
   2. Update thesis documentation and make it available online: Umar
6. Analysis
   1. Provide API (agreed 6/28/07, changes to TULIP mean current API is not working):
   2. Literature search on triangulation, multi-lateration:
   3. Optimize alpha by landmark (after have new location algorithm working, will try 1:2 for US, 1:3 for Europe, 2:15 Africa, done at NIIT, need to deploy at SLAC 4/10/07, Faran is worried about how to group nodes by region etc. and anomalies 6/13/07), hope to review results by 6/5/07): Faran
   4. Map TULIPs accuracy by region. This will help in identifying where to add new landmarks : Shahryar
   5. Analyze Speedtest sites with and without PlanetLabs landmarks. Also study results for North America using all 150 sites, will report results by 03/16/08: Shahryar
      1. Determine which sites have the highest frequency of being selected as one of the top 3 landmarks.
      2. Observe why some nodes are showing results with errors greater than 1000 Km.
      3. (Data used for analysis should be available to everyone (to allow them to conduct analysis)
      4. Why isn't SLAC listed in the list of sites?
      5. Why in the worst cases, the landmarks in the vicinity of the target failed to respond?
      6. Why do the results worsen (in some cases) when using both Planet Lab and PingER nodes?
      7. What values of alpha to use for each tier?
      8. Does the alpha value vary with increase in distance between the landmark and the target, how much and why?
      9. Document the findings
   6. Compare with GeoIP Tool & Hostip.info
      1. Find cases where TULIP beats GeoIP Tool [Done 4.68.116.16, probably backbone routers too]
      2. Compare with MaxMind (see http://www.maxmind.com/app/geolitecountry): Faran
      3. Document comparison in Wiki (is this done?): Shahryar
      4. Confidence estimates of accuracy
      5. Applied to PingER hosts
      6. Outside triangles, area coverage
      7. Effect/need to vary alpha: Faran
      8. Look at big discrepancies
   7. Provide bubble maps of accuracy: Shahryar
   8. What area/population is covered by TULIP
   9. Show accuracy by region and country: Shahryar
7. Look at publication
   1. Find conference: all
   2. Can we get Cisco interested?: Arshad, Faran
   3. Update documentation: Les, Faran
   4. Read paper at http://research.microsoft.com/%7Epadmanab/papers/sigcomm2001.pdf:
   5. Put Faran's final year project documentation online: Faran, Umar

Netflow

1. Frontend - Yee, Ahmad
   1. Offer suggestions on interface improvements (ahmad)
   2. Fix incompatibility with IE6,7 (Ahmad)
   3. Javascript documentation - non currently exists, inline with javascript
   4. spider & pie charts (Shahryar)
      1. Tidy up problem with legends
      2. implement PNG functionality
      3. Documentation on wiki
   5. Implement and test distribution mechanism for frontend (web) code into installation scripts (Ahmad)
   6. Implement web frontend as virtual host for apache (Ahmad)
   7. RRD probs
      1. Get Yee an account so he can look around on Ahmed's host to help understand problems.
2. Backend (Akbar, Yee)
   1. Make scoreboarding more useful in terms of categorising under subnets, application and countries.
   2. RRD file organisation - refactor code to enable full paths in rrd files and also /source/dest/source-dest_<type>.rrd file location.
   3. Test Ahmed's new installation script (Akbar)

PerfSONAR

1. Middleware
   1. Document User API for perfSONAR: Asif, Qasim
      1. how does this differ from perfsonarui?
   2. Study the perfSONAR NMWG request/response schema [Done 5/1/07]: Qasim
      1. Document NMWG request/response schema on wiki (append to existing docs): Qasim
   3. Output from some script (mtml.pm?) is unintelligible (e.g. control characters) need help from Asif, Nauman will send snapshot o Asif (5/10/07)
   4. Having Java socket error, maybe firewall problem: Asif
   5. Add updated code into SVN: Asif
2. Make traceroute.pl (reverse traceroute and ping server) perfSONAR compliant.
3. Visualisation
   1. Provide 3-dimensional spinning (interactive) globe interface like with mona-lisa in java 
      1. integrate into Asif's topoviz frontend: QasimB
         1. Will send screen shots (7/26/06): QasimB
         2. Consider GoogleEarth for 3D Topology visualization: QasimB
      2. integrate into perfsonarUI: TBA
         1. Map relevant links & paths perfSONAR node information (see https://svn.internet2.edu/svn/nmwg/trunk/nmwg/schema/rnc/topo for schema) .
         2. Integrate into perfSONAR libraries Asif was working on
            1. Get documentation on perfSONAR user libraries: Asif
      3. Make useful for PingER?
         1. Display mon & remote sites
         2. Mouse overr provides site details
         3. Mouse click on monitor provides sites-per-country.html like output for monitor node
         4. Select metric & ticks src dest submit pingtable request and get plot.
   2. Googlemaps 
      1. Download and understand code from internet2 svn
      2. install locally
      3. Investigate google maps API's and extensions to provide 'traffic' like visualisation (look at google maps at the Bay Area for example)
   3. Ping summary table: Nauman
      1. status? (was first thing ever assigned to nauman)

PerfSONAR-UI

1. General: Qasim Bilal, Yee
   1. Study the link and documentation of PerfSONAR (available online): Qasim
   2. Application Requirements
      1. Get the data from MA for Pinger
      2. Parse it to show mesh
      3. Draw utilization graph for site(s) selected
      4. Check for schema deficiencies
2. PerfSonar Java Libraries for Backend: Qasim
   1. PerfSONAR user libraries - i.e. libraries that will make it easy for others to gather and consume data from perfsonar services. the specific requirments are to integrate the Topology Service, Lookup Services, the RRD/SNMP MA's, and Status MA's
      1. Using XQuery or templating library query the schema.
      2. Study and understand XQuery
      3. Implement it using the schemas given here
3. 3D User Interface for Pinger and Perfsonar: Qasim, Les, Yee
   1. Create nodes (with colour/icons) and links (with either arrows or a moving animation of direction, with colour)
   2. Automated spinning
   3. Zooming and panning
   4. Conversion from 3D Globe to 2D image
   5. Integration of this code with PerfsonarUI and also maintain separate copy.

Pathneck (see http://maggie.niit.edu.pk/website/projects/Topological_Analysis.html) Yee is the lead at the SLAC end: Asif

THIS IS OLD AND DEFUNCT, WE ARE CURRENTLY NOT INVESTIGATING INTO THIS ANYMORE

1. Try and get pathneck working from NIIT, study debug output, work with Ningning <hnn@cs.cmu.edu> (awaits site upgrade): Asif
2. Update production version at SLAC (12/106): Asif, Yee
3. Develop and install pathneck reverse server (would like to start at NIIT just to assure code works work with -y option): Les, Asif
   1. Modularize security of traceroute.pl: Asif, Yee
   2. Add in PerfSonar: Asif, Yee
4. Evaluate and understand how pathneck needs to be configured, the failure modes, the areas of applicability etc. using iepm-resp.slac.stanford.edu: Asif
   1. Provide web site documenting failure modes: Asif
5. Develop a vizualization
   1. Topology (currently with Touchgraph), color links, mouseover ID router, provide snapshot
   2. Click on link/router to get graph
   3. Incorporate location (e.g. via TULIP)
6. Need a schema
7. Investigate integration with traceanal and graphviz: Asif
8. Integrate topology map with pathneck: Asif
9. Look at using PerfSONAR router information (do not agree well, need to get more statistics, to enable multiple measures need API to get PerfSONAR data, look perfsonar UI): Asif
10. Try on Ultralight (problems if < 4 hops): Asif
11. Problem of 0 results (reported to author but not much happening 4/20/06): Asif
12. Add position and AS information to router database: Asif

Communications etc.

1. Projects (see http://www-iepm.slac.stanford.edu/pinger/projects/)
2. Resolve next graduates/RAs/visitors: Arshad, Les
   1. Umar tentative arrival Feb 17, 2008
   2. Next student probably in July/Aug 2008
   3. Umar is hiring RAs (3 people for MAGGIE for 2 years). Shahryar 7 Asif will apply, could be 6 or 7 people.
3. Paper for Nepal can't get travel funding, investigating whether will accept paper without attendance: Shahryar
   1. Funding for registration fee acquired. Paper has been registered. Waiting for title of the proceedings and access to an online copy: Shahryar
4. Paper on event diagnosis: Adnan
   1. Future plans Needs improving: Adnan
   2. There are now (1/12/08) 2 students working on diagnosis, and one on Network Weather Forecasting. If they turn out well, they will be introduced to SLAC.
   3. Hope for something in a few months
5. Proposals
   1. HEC funding approved
   2. USAID proposal due 6/30/07### Troubles have delayed review from November to January
   3. Proposal to Cisco on TULIP:  Shahryar
   4. Proposal to Cisco on perfSONAR (sent, await response): Les
6. Progress Reports from NIIT
   1. Customized the net-weekly.pl script and deployed at NIIT (03/03/08): Umar
   2. Students to maintain rough drafts of their project descriptions at NIIT's wiki
7. WIKI Account at SLAC
   1. Couldn't find details of resetting the password for shahryar's (confluence) wiki account. Applied for a new account.