Xrootd Setup for Glast

The glast xrootd setup uses the standard setup for the xrootd installation and log directories:

This page describes the xrootd setup for GLAST.
The GLAST xrootd cluster consists of redirectors and data server. A redirector is the central entrance point for a client.

Activating a xrootd version

Taylor installs various xrootd releases into /opt/xrootd of a server. The activate script to enable a particular version. The activate script runs taylor and then creates the /opt/xrootd/prod link that points to the xrootd release to be used. All other scripts (start, stop, ranger) are using the /opt/xrootd/prod path to find binaries and config files.

activate <xrootd-version>

For example:
    > cd /opt/xrootd
    > activate  20070321-1251p1;

Start/Stop the xrootd/cmsd daemons

The glast xrootd servers (data servers and redirectors) are configured so that ranger will automatically restart a non running xrootd or olbd daemon. Ranger runs every 15 min.

The prod/etc directory contains scripts to stop and start the daemons. Most of the times it is the easiest to restart both xrootd and cmsd daemons using the RestartALL script.

> cd /opt/xrootd
> ./RestartALL     # Restart xrootd and cmsd

On a redirector server only a cmsd and xrootd daemon are running.
On a data server, a few more processes are started depending on the configuration.
For the configuration used by GLAST one should see that the cmsd spawns two processes:

• XrdOlbMonPerf collects load statistics that is reported to the redirector cmsds
• ooss_prep command used to prestage files

Cache file systems 

Typically data servers employ more then one file system. In order to provide a single name space Xrootd uses the the cache system provided by the ooss (mps) tools. A cache directory is created for each file system. Files are placed into these cache directories and a link is created from the so called namespace to the file in the cache. The example will illustrate this:

1. Lets assume there are two file systems.
2. The two directories
   /glast/cache0   and  /glast/cache1
   exists for the two file systems.
3. We copy the file /glast/mc/DC2/test1/0001/02/test1_digi_DIGI.root  into xrootd
   1. xrootd figures out which cache to use, lets assume /glast/cache1
   2. The file is copied to:
      /glast/cache1/%glast%mc%DC2%test1%0001%02%test1_digi_DIGI.root
   3. A link in the namespace is created:
      > ls -l /glast/mc/DC2/test1/0001/02/test1_digi_DIGI.root
      /glast/mc/DC2/test1/0001/02/test1_digi_DIGI.root -> /glast/cache1/%glast%mc%DC2%test1%0001%02%test1_digi_DIGI.root
   4. In addition Xrootd migth create a .lock file if it is configured to allow migration.
      > ls -lL /glast/mc/DC2/test1/0001/02/test1_digi_DIGI.root*
         -rw-rw-r--   1 glastdat gl       157558240 Jan 12 01:11  /glast/mc/DC2/test1/0001/02/test1_digi_DIGI.root
         -rw-rw-r--   1 glastdat gl       157558240 Jan 12 01:11  /glast/mc/DC2/test1/0001/02/test1_digi_DIGI.root.lock
      The time stamp of the file and the .lock file is used to determine if a file has been migrated. (if file is newer than .lock means that it has not yet been migrated)