We use the "tulip" database to generate our sites.xml which is further used in probing the landmarks. We added pingER nodes from the Nodedetails database to the tulip database but with some defined rules. We only added those nodes which contain a traceroute server. To implement this and make it working we developed three packages.
- TULIP/ANALYSIS/NODEDETAILNODES.pm
- insert_sites-xml.pl
- create_sites-xml.pl
TULIP/ANALYSIS/NODEDETAILNODES.pm
In order to build this module we used some predefined perl modules and scripts. To get data from node details database we used require '/afs/slac/package/netmon/pinger/nodes.cf' and we used standard package Text::CSV_XS to convert our data in comma separated values. In order to define our node to be a candidate for a tulip landmark we tested it for a few conditions which include that the node must have traceroute server, it should not be set to NOT-SET and it's project type should not be set to "D" which means deleted as per nodedetails database semantics. The nodes which qualified these conditions were put into a separate array. This array is used by insert_sites-xml.pl to insert these sites to Tulip database
insert_sites-xml.pl
This perl script is used to create the insert query from the data of above nodes. Again using the perl package Text::CSV_XS we divide our data into separate chunks. The data is then fed to the structure which contains parameters for the query. This script resolves each hosts with host names taken from nodedetailnodes.pm this helps in eliminating bad hosts if they exist. Before inserting new nodes into database it checks weather the node is in nodedetail or not. We use ipv4Addr as our unique key. We traverse through the tulip database and check if there exists some node with the same ipv4addr if it exists we ignore the entry and if not we go ahead with inserting it in the database.
create_sites-xml.pl
This perl script is used to create the sites.xml file which is further used in our TULIP project as a source for node information and landmarks. This perl module uses the template library in order to generate required xml. It traverses through tulip database and gets each node, checks the service type and generates the file with all the available parameters in the database.
Tulip Transition to Sites.XML
Our next step in the process is to transform TULIP so that it can get data from created sites.xml. TULIP version 1 was having two different data sources one to get data from nodedetail data base and the other was to fetch data from a list containing Planet lab sites.
Inorder to perfom this step we used JAVA Xerces API , which is described as the best option in terms of resources used and efficiency in parsing XML data. We used SAX model because of following reasons.
1) The list of landmarks would increase as more pinger monitoring sites are added
2) We donot require to rewrite or add xml to the document from TULIP
Tutorial posted by SUN is helpful in understanding parsing of XML. The tutorial can be found over here
TULIP Client
TULIP has a Java based client which have the important functions and classes for sending the query over web to reflector.cgi. In addition to sending query and getting/parsing responses TULIP client uses the algorithm to find the location of queried host. This section explains technical details about the tulip client and the classes used and important function. The re-implementation of TULIP has led to proper packages and use of ANT package management tool by Apache for code compilation.
Depandancies
Current implementation of TULIP Java client depend on two files for its execution
- Sites.xml
- /afs/slac/package/netmon/tulip/Initial.txt
The former is dicussed above in this article. The latter is used to Automate test. Its a list of nodes/sites which TULIP tries to locate when it is run from command lin.
Code Deployation at SLAC
TULIP is deplyed at SLAC in /afs/slac/package/netmon/tulip/src
The class hirarichy is defined in three folders.
- Library Folder
- The lib directory contains third party packages used for implemenation of TULIP. For eg we are using Java Matrix Class which provides the fundamental operations of numerical
linear algebra.
- The lib directory contains third party packages used for implemenation of TULIP. For eg we are using Java Matrix Class which provides the fundamental operations of numerical
- TULIP Folder
- This folder contains actuall implementation of TULIP client. It has two branches core and utill. The core package contains core classes or classes which are used in directly invloved in implementation of Tulip. The utill package contains all the utillity classes for the core classes i.e they help in implementation. For instance to get the data we need to parse sites.xml. So, getData is core class where as xml parsing class is helping it to achieve its goal so its in utill package.
- This folder contains actuall implementation of TULIP client. It has two branches core and utill. The core package contains core classes or classes which are used in directly invloved in implementation of Tulip. The utill package contains all the utillity classes for the core classes i.e they help in implementation. For instance to get the data we need to parse sites.xml. So, getData is core class where as xml parsing class is helping it to achieve its goal so its in utill package.