You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 9 Next »

Using a release

The xrootd tools used by FGST are in the release directory of the glastdat user. Sourcing the wk_env script will set the PATH to a release bin directory and set the PYTHONPATH for the python packages.

 % cd /u/gl/glastdat/releases/admin/
 % . bin/wk_env

Running commands on remotely on servers

The ScaRunOnHosts script will take a list of FGST xrootd servers and a command. It will ssh to all of the servers in the list and execute the command directly on the server. The basic usage is:

  % ScaRunOnHosts -s wain018,wain034 -r -e -- <command>
  % ScaRunOnHosts -f glast -r -e -- <command>

It should be run from a release as described above. The available options are:
-p : ssh to all host in parallel
-w : wait for all hosts do be done. Only useful with -p option.
-e : Setup the release environment on the remote host. With out this option the
command to be run has to be found in the default PATH of the glastdat
-s srv1,srv2,..,srvn : list of servers to run the command
-f <flist> : read server list from a file. The file is looked for in /afs/slac.stanford.edu/g/glast/applications/xrootd/config/

Remove old recon files SrvMpsPurge.sh

The SrvMpsPurge.sh script is use to remove old recon files from a xrootd server. The script has hard coded into it the location of the list with the old recon files. It employs another script (mps_purge.py) that does the work. Purging a file includes the following steps:
#) Check if file is on disk
#) compare files mtime with the one of the corresponding .lock file to find out if the file has been migrated to HPSS.
#) If the file is migrated check in HPSS if the file exists and compare file sizes.
#) remove file (and .lock file) if the check in HPSS succeeded.

The purge script works only on Solaris servers as they use .lock files to indicate the migration status. The Linux servers are using extended file attributes.

SrvMpsPurge.sh accepts a few options (all optional)

  -l   write output to log file
  -n   number of files to purge
  -T   test, print only command and exit

A typical example to run the purge on all xrootd Solaris (wain, sulky) servers is:

 % cd /u/gl/glastdat/releases/admin/
 % . bin/wk_env
 %  ScaRunOnHosts -f glastsol -p -w  -e -r --  SrvMpsPurge.sh -l

The above command will ssh to all Solaris xrootd servers and run the SrvMpsPurge.sh script. Log files are written to the directory /var/adm/mps/logs and the logfile name is rm_file.YYYYMMDDTHHMMSS. As all output from SrvMpsPurge.sh_ is written to a log file one will not see any output from the command above.

Running only on a single server run and purge no more then 20 files:

 % cd /u/gl/glastdat/releases/admin/
 % . bin/wk_env
 %  ScaRunOnHosts -s wain034 -e -r --  SrvMpsPurge.sh -l -n 20

Migration to HPSS

Migrate files using the daily file lists

ScaRunOnHosts -f glast -e -r -- SrvMigrateFiles.sh -b [-r] [-e]

The -r migrates the recon files, -e use the extra-backup file lists.

re-migrate a file

  1. Create a file, <file-list>, with filenames that need to be remigrated (/glast/....)
  2. Mark file on server as migratable (Linux): 
     frm_admin mark -m /glast/Data/.....
  3. Run migration 
    SrvMigrateFiles.sh  <file-list>

Migration commands

Show the current migration status

ssh to all data server and tail the last few lines of the most recent migrate log file. Count the number of migrated files.

 ScaRunOnHosts -f glast -e -r -- SrvMigrStat.sh

Stop migration using semaphore file

The migration tools check for a semaphore file (/var/adm/frm/STOP_MIGR or /var/adm/mps/STOP_MIGR) and will exit if it exists. To set remove or list the semaphore files on all servers run:

 ScaRunOnHosts -f glast -e -r --SrvMigrateFlags.sh [list | rm | set]
  • No labels