Background
PingER uses a database for the contact and geographical node locations of various pinger sites around the world. The idea of this perfsonar topology service for pinger will enable the querying of such data and the dynamic discovery of it.
Note that this page does not describe the database itself (which will be changed later to better accomodate the future).
Topology Services
The perfSONAR PS Topology Service was designed primarily to export data regarding the inter-relations between network paths/links. However, a subset of the data enables the representation of node information which fits in nicely to that provided by the current PingER 'nodedetails' database.
Therefore, it seems like a nice way of easily exporting PingER node information through an existing service.
This process requires the following steps:
- mapping of the pinger database fields to xml topology elements
- processing of the above into a single 'store' file that contains a snapshot of the topology (a list of nodes in our case for pinger)
- deployment and configuration of the topology service
- loading of the 'store' file into the topology service
- testing of the topology service
Field mapping
Unfortunately, there are various nuances with the way that the nodedetails table is formatted. Most of the mappings and regex's are to deal with these 'features'.
Generally, the mapping can be expressed from teh following segment of perl code (i'm to lazy to write a proper document)
my $node = {
'id' => $nodename,
'domain' => $row->{SITENAME},
'hostName' => $row->{NODENAME},
'name' => $nodename, # strip out domain
'location' => {
'institution' => $row->{FULLNAME},
'country' => $row->{COUNTRY},
'city' => $row->{LOCATION},
'state' => $state,
'longitude' => $long,
'latitude' => $lat,
},
'contact' => \@contacts,
'port' => {
'ipAddress' => $row->{IPADDRESS},
},
'comments' => $row->{COMMENTS},
};
$nodename is the host name of the node, eg iepm-bw.slac.stanford.edu would have a nodename of iepm-bw and a domain of slac.stanford.edu Note that a domain may be arbitrary in structure (we could have chosen iepm-bw.slac as the nodename and stanford.edu as the domain - this needs better clarification by the perfSONAR guys.
The longitude and latitude has to be parsed from the single field in the nodedetails table.
The contacts is an array of contact xml elements, which were passed with the following:
foreach my $info ( split /,/, $row->{CONTACTS} ) {
if ( $info && $info =~ /\s*(.*)\s*\<(.*)\>/ ) {
# if the email address has the same sitename as the node, then assign the same institution name
my $contactInst = undef;
if( $2 =~ /$row->{NODENAME}/ ) {
$contactInst = $row->{FULLNAME};
}
my $contact = {
'priority' => $i,
'administrator' => $1,
'email' => $2,
'institution' => $contactInst,
};
push @contacts, $contact;
$i++;
} #if
} # foreach
The complete parsing script can be found at:
svn: https://svn.internet2.edu/svn/perfSONAR-PS/trunk/perfSONAR-PS/MA/PingER/pinger_topology.pl
The script reads in the database table nodedetails and returns the relevant xml data. It maps the above perl hash is mapped to a XML template file of the format:
<nmtb:topology
xmlns:nmtl2="http://ogf.org/schema/network/topology/l2/20070707/"
xmlns:nmtl3="http://ogf.org/schema/network/topology/l3/20070707/"
xmlns:nmtl4="http://ogf.org/schema/network/topology/l4/20070707/"
xmlns:nmtb="http://ogf.org/schema/network/topology/base/20070707/">
[% FOREACH node = nodes %]
<nmtb:node id="urn:ogf:network:domain=[% node.domain %]:node=[% node.id %]">
<nmtb:name type="string">[% node.name %]</nmtb:name>
<nmtb:hostName>[% node.hostName %]</nmtb:hostName>
<nmtb:description>[% node.description %]</nmtb:description>
[% FOREACH contact = node.contact %]
<nmtb:contact priority="[% contact.priority %]">
[% IF contact.administrator %]<nmtb:administrator>[% contact.administrator %]</nmtb:administrator>[% END %]
[% IF contact.email %]<nmtb:email>[% contact.email %]</nmtb:email>[% END %]
[% IF contact.phoneNumber %]<nmtb:phoneNumber>[% contact.phoneNumber %]</nmtb:phoneNumber>[% END %]
[% IF contact.institution %]<nmtb:institution>[% contact.institution %]</nmtb:institution>[% END %]
</nmtb:contact>
[% END %]
<nmtb:location>
[% IF node.location.institution %]<nmtb:institution>[% node.location.institution %]</nmtb:institution>[% END %]
[% IF node.location.country %]<nmtb:country>[% node.location.country %]</nmtb:country>[% END %]
[% IF node.location.zipcode %]<nmtb:zipcode>[% node.location.zipcode %]</nmtb:zipcode>[% END %]
[% IF node.location.state %]<nmtb:state>[% node.location.state %]</nmtb:state>[% END %]
[% IF node.location.city %]<nmtb:city>[% node.location.city %]</nmtb:city>[% END %]
[% IF node.location.streetAddress %]<nmtb:streetAddress>[% node.location.streetAddress %]</nmtb:streetAddress>[%
END %]
[% IF node.location.floor %]<nmtb:floor>[% node.location.floor %]</nmtb:floor>[% END %]
[% IF node.location.room %]<nmtb:room>[% node.location.room %]</nmtb:room>[% END %]
[% IF node.location.cage %]<nmtb:cage>[% node.location.cage %]</nmtb:cage>[% END %]
[% IF node.location.rack %]<nmtb:rack>[% node.location.rack %]</nmtb:rack>[% END %]
[% IF node.location.shelf %]<nmtb:shelf>[% node.location.shelf %]</nmtb:shelf>[% END %]
[% IF node.location.longitude %]<nmtb:longitude>[% node.location.longitude %]</nmtb:longitude>[% END %]
[% IF node.location.latitude %]<nmtb:latitude>[% node.location.latitude %]</nmtb:latitude>[% END %]
</nmtb:location>
<nmtl3:port id="urn:ogf:network:domain=[% node.domain %]:node=[% node.id %]:port=[% node.port.ipAddress %]">
<nmtl3:ipAddress type="IPv4">[% node.port.ipAddress %]</nmtl3:ipAddress>
</nmtl3:port>
[% IF node.comments %]<nmtb:comments>[% node.comments %]</nmtb:comments>[% END %]
</nmtb:node>
[% END %]
</nmtb:topology>
Creation of 'store' file
running the following will generate the xml topology store file:
[ytl@pinger:~/Work/perfSONAR/perfSONAR-PS/trunk/perfSONAR-PS/MA/PingER]$ ./pinger_topology.pl > pinger_topology.xml
An example snippet of the xml is here:
<nmtb:topology xmlns:nmtl2="http://ogf.org/schema/network/topology/l2/20070707/"
xmlns:nmtl3="http://ogf.org/schema/network/topology/l3/20070707/" xmlns:nmtl4="http://ogf.org/schema/network/topology/l4/20070707
/"
xmlns:nmtb="http://ogf.org/schema/network/topology/base/20070707/">
<nmtb:node id="urn:ogf:network:domain=lanl.gov:node=www">
<nmtb:name type="string">www</nmtb:name>
<nmtb:hostName>www.lanl.gov</nmtb:hostName>
<nmtb:description></nmtb:description>
<nmtb:location>
<nmtb:institution>Los Alamos National Laboratory</nmtb:institution>
<nmtb:country>United States</nmtb:country>
<nmtb:state>NM</nmtb:state>
<nmtb:city>Los Alamos</nmtb:city>
<nmtb:longitude>35.73</nmtb:longitude>
<nmtb:latitude>-105.15</nmtb:latitude>
</nmtb:location>
<nmtl3:port id="urn:ogf:network:domain=lanl.gov:node=www:port=204.121.x.y">
<nmtl3:ipAddress type="IPv4">204.121.x.y</nmtl3:ipAddress>
</nmtl3:port>
</nmtb:node>
<nmtb:node id="urn:ogf:network:domain=cnea.gov.ar:node=eros">
<nmtb:name type="string">eros</nmtb:name>
<nmtb:hostName>eros.cnea.gov.ar</nmtb:hostName>
<nmtb:description></nmtb:description>
<nmtb:location>
<nmtb:institution>Tandar/CNEA</nmtb:institution>
<nmtb:country>Argentina</nmtb:country>
<nmtb:city>Buenos Aires</nmtb:city>
<nmtb:longitude>-34.617</nmtb:longitude>
<nmtb:latitude>-58.400</nmtb:latitude>
</nmtb:location>
<nmtl3:port id="urn:ogf:network:domain=cnea.gov.ar:node=eros:port=168.96.x.y">
<nmtl3:ipAddress type="IPv4">168.96.x.y</nmtl3:ipAddress>
</nmtl3:port>
<nmtb:comments>[Beacon - Argentina monitoring, add 12/8/0] (Tandar - Particle Accelerator/CNEA, Centro Atomico Constituyentes
(CAC), Buenos
Aires campus, Argentina)</nmtb:comments>
</nmtb:node>
...
(note some ip;s were stripped for security)
Deployment and configuration of Topology Service
Currently the various perfSONAR services are still under development, so this step should be a lot simpler in the future (not that it isn't now)
The service will be deployed on
http://lhcopnmon1-mgm.fnal.gov:8083/perfSONAR_PS/services/topology
Check out source from SVN
We will create a directory called perfSONAR and checkout the current snapshot to it.
[iepm-bw@lhcopnmon1-mgm ~]$ mkdir perfSONAR [iepm-bw@lhcopnmon1-mgm ~]$ cd perfSONAR/ [iepm-bw@lhcopnmon1-mgm perfSONAR]$ /usr/bin/svn co https://svn.internet2.edu/svn/perfSONAR-PS .
After a bit of time, all of the snapshot will be put inside.
The directory that we are interested in is at:
./branches/merge/services/Topology
Configuration
There is a config file available to automate much of the install:
[iepm-bw@lhcopnmon1-mgm Topology]$ ./configure.pl -- perfSONAR-PS Topology MA Configuration -- - [press enter for the default choice] - What file should I write the configuration to? [topology.conf]: Enter the listen port [8083]: Enter the listen end point [/perfSONAR_PS/services/topology]: Enter the maximum number of children processes (0 means infinite) [4]: Enter number of seconds a child can process before it is stopped (0 means infinite) [0]: 180 Enter the directory containing the XML Database (if relative, it's relative to the installation directory) [xmldb]: Enter the filename for the XML Database [topology.dbxml]: Is this service read-only? [1]: Will this service register with an LS [1]: Enter the number of minutes between LS registrations [30]: URL of an LS to register with [http://perfSONAR2.cis.udel.edu:8181/perfSONAR_PS/services/LS]: http://patdev0.internet2.edu:6666/perfSONAR_PS/services/LS Enter a name for this service [Topology MA]: Enter the service type [MA]: Enter a service description [Topology MA]: External hostname for this service []: External hostname for this service []: lhcopnmon1-mgm Writing topology.conf: done. Checking dependencies for desired configuration... mkdir /root/.cpan: Permission denied at /usr/lib/perl5/5.8.5/CPAN.pm line 2342
Needs root. dont' have it. Stalled.