...
The archiver is another system that works on top of the Workflow system. Just like the Release Manager, it has a trigger script registered with the workflow system that triggers when a new archive job is started. Similarly, the archiver contains database entries and a web page for viewing the information.
Database
The database for the Archiver is stored on glastDB.slac.stanford.edu with the database name of Archive. It contains the following tables:
- archiveContent - The files archived.
- pending - Contains pending archive entries.
- tarFiles - Contains a list of tar files that belong to a single archive job and their locations in mstore/gstore/astore.
- task - The main table containing the archive job.
All the tables are tied together via an id field defined by the task table. Tar files are split at a size defined by the user at archive creation time and the list of tar files belonging to a single task is stored in the tarFiles table.
Scripts
The scripts for the Archiver are stored in /u/gl/glast/infraBin/Archiver. The scripts located there are:
- archive.pl - The script invoked by the workflow when archiving.
- archiver.pl - This script is the controller script with which users can archive, restore, delete jobs.
- delete.pl - This script is invoked by the workflow when deleting archived files.
- determineTask.pl - This script is invoked to determine if a job is for archiving, deleting, restoring, etc.
- finish.pl - This script is invoked as the last script by the workflow to cleanup the database and mstore/gstore/astore.
- restore.pl - This script is invoked for restore operations.
- trigger.pl - This script is invoked by the workflow to determine if new archiving tasks are pending.
- verify.pl - This script is invoked for verify operations.
All of these scripts are mostly frontends to the mstore/gstore/astore applications. The use the expect perl module to programmatically control tar which expects interactive input for creating and splitting tar files.
Here is an example how to archive a file CLHEP-1.9.2.2.tar.gz in directory /nfs/farm/g/glast/u05/extlib/tiger_gcc33:
/afs/slac.stanford.edu/u/gl/glast/infraBin/Archiver/archiver.pl --module "Manual" --callback "user:kiml" --path "/nfs/farm/g/glast/u05/extlib/tiger_gcc33" --user glast --name "u05.extlib.tiger_gcc33.CLHEP-1.9.2.2.tar.gz" --file CLHEP-1.9.2.2.tar.gz --method mstore archive
And to restore it to /nfs/farm/g/glast/u30/tmp:
/afs/slac.stanford.edu/u/gl/glast/infraBin/Archiver/archiver.pl --module "Manual" --callback "user:kiml" --path "/nfs/farm/g/glast/u30/tmp" --user glast --name "u05.extlib.tiger_gcc33.CLHEP-1.9.2.2.tar.gz" --file CLHEP-1.9.2.2.tar.gz --method mstore restore
Notification
The RM scripts notify the users of problems encountered during checkout, compile, or unit tests. This notification is done by the finish.pl script. It checks the database for every package that belonged to a build to determine if the package had any errors during checkout, build, or unit test compiles. If there were errors, the script checks who the author is for the failure and notifies the author. Additionally, all the failures are accumulated and sent to a general failure mailing list as specified by the settings page.
Webpages
The web page for the Archiver is https://www.slac.stanford.edu/www-glast-dev/cgi/DiskArchive. It is controlled by the SLAC web server. The information is displayed by a bunch of perl scripts located in /afs/slac/g/www/cgi-wrap-bin/glast/ground/cg/archive.
Old Style Tagger
With the shut down of automated CMT builds, the old style tagger has also been shut off (June 2013) and its use is now deprecated. However, the material is kept here for reference.
...