Page History

Purpose

 The The main purpose of TULIP Central Reflector is to proxy the TULIP queries to PlanetLab's Scriptroute Service. It may also be extended to issue all queries.  This decision will be related to speed of execution and security among other things. The PlanetLab Scriptroute service provides a cookie which works for a single IP address only. So in this way all the requests will be issued from the Central reflector and the responses will be sent back to the TULIP JNLP Client. Here is a map of Planet Lab servers.

Implementation 

 The TULIP Central reflector will be a CGI Script (reflector.cgi) deployed at SLAC. The TULIP client will issue a single request and the Reflector will go ahead and probe all the landmarks in that region*\[1\] and return the results to the TULIP Client. Probing the target site from more vantage points may give us a better estimate of its location.

Requirements

• Should fetch sites.txt or have a local copy of sites.txt, what changes should be made to sites.txt ?
  • A new parameter should be added to sites.txt to include teir0 or teir1. Also the region of tier1 sites needs to be specified in sites.txt
• A separate thread should be used for each landmark and Semaphores should be used for locking, so that data from different threads should not inter-mix.
• There should be a limit on the number of threads that can be launched at a time (say 10).
• Should there be extra logging on the reflector or can we rely on the standard web logs which will log each query including time stamp, the client name.  What else it logs depends on whether the rewuest is Get or a Post.
• Where are the results parsed, could be in the reflector or in the Java client. In the client distributes the parsing load, reduces the load on the reflector, simplifies the CGI script.
• What should happen if a landmark responds with a bad data. ( Should it process the error or send the raw data back?). Since there will be some anomalies I suspect the reflector will need to return the full response and anyhow needs to inform the user, so I suspect initially the clinet will process the response and spot errors etc.) Also if the client parses the result it will probably be easily able to spot problems.
• Special consideration for security as the script ultimately has to be deployed at SLAC ( Perl taint option, warning option, special open method etc)
• Need to agree on a common format for the exchange of data.
• Needs a blacklisting mechanism for malicious hosts.

 After After discussing with Yee and Booker it was seen that forks may be too complicated. The version of Perl at SLAC did not support threading. Also the security people will not allow forks running inside a CGI-script. So I had to come up with an alternative. The solution to this problem was to use Asynchronous IO.  A bunch of requests could be send to the landmarks without waiting for he response.  The  LWP::Parallel library provides all this functionality.  It supports asynchronous IO. Currently it is not installed  so I am using a local version  in my home directory.  Ultimately this module has to be installed on the production server.

  I have implemented most of the functionality. The script is running fine.  I will have to taken measures to make the script more secure so that it could not be used as a platform to launch DDOS attacks, by limiting the number of concurrent process of reflector.cgi to 10. Also the script produces customized messages such as ( request time out or Connection failed so that TULIP client can differentiate between the various kind of error conditions). Also there is a blacklisting mechanism so that particular IP addresses can be blocked.

...

For the Planetlab landmarks an interpretive script (also see here for the original is supplied with the target ($target) and the number of pings ($ping) to make. For the SLAC ping servers and the Looking glass sites the landmark is accessed by a URL provided in the sites.txt file (see below) in the PingSites token (e.g. http://www.slac.stanford.edu/cgi-wrap/nph-traceroute.pl?choice=yes&function=trace&target=$target).

Files

New XML landmarks file

TULIP has been refactored to use an XML file for its site/landmark information. This is generated from a MySQL database in the 'tulip' schema on pinger. The file can be generated with a perl script in the subversion repository, pinger/trunk/bin/create_sites-xml.pl and its associated template file, pinger/trunk/bin/sites-xml.tt2. Add/modify nodes in the sites list by connecting to the database.

Old landmarks file

The list of SLAC and Looking Glass landmarks is read from /afs/slac/www/ The list of SLAC and Looking Glass landmarks is read from /afs/slac/www/comp/net/wan-mon/tulip/sites.txt. the format is space separated tokens:

SNo Site_name PingSite TraceSite Lat Long Reference_Site Alpha, for example (where \ means line broken for viewing)

...

1  SLAC,Stanford_US http://www.slac.stanford.edu/cgi-wrap/traceroute.pl?function=ping&target=
\ http://www.slac.stanford.edu/cgi-wrap/nph-traceroute.pl?choice=yes&function=trace&target=
\  39.32  -122.04 www.slac.stanford.edu   73

 The The list of PlanetLab landmarks is read from: /afs/slac.stanford.edu/www/comp/net/wan-mon/tulip/TULIP/newsites.txt. It appears as:

...

Anything following a # sign is ignored (it is a comment).

 The The tokens are space delimited, they are:

Next version

This is still in design/discussion. We want to remove, where possible, the overloading of tokens, make it more amenable to adding new tokens, and tie in better to the pingER NodeDetails database. Our first stab at the template for the new XML file can be found here. An initial version was created here from the original PlanteLabs and SLAC sites.txt files at:

my @files=("/afs/slac.stanford.edu/www/comp/net/wan-mon/tulip/newsites.txt",#PlanetLabs
             "/afs/slac.stanford.edu/www/comp/net/wan-mon/tulip/sites.txt",   #SLAC

We will also take this opportunity to clean up the data. For example make sure the city and country appear and the the country and region are spelt correctly (see Conventions below), that elements are in the right order etc. 

The intent is that the tier and alpha values will be added to the NodeDetails database later and this will then become the authoritative source from which the XML will be created.

It is unclear whether we need to add alpha or whether this should be determined by the program itself, so we need a value to decide whether to leave it up to the program.

...

Conventions

We use the following conventions:

...