Backup of Xrootd FGST data to HPSS
The FGST xrootd cluster is integrated with HPSS. New files that are copied into the xrootd cluster are marked that they have not been migrated to HPSS. Not all files that are copied to xrootd are backed up but currently only some of the L1 produced files. The list of files that should be migrated to HPSS is produced daily and is used by the migration tools to transfer files to HPSS. Once a file has been migrated to HPSS it is marked as migrated so that it will not be migrated again.
There are three lists of L1 files to backup, found in ~glast/_backupfiles. Each line of these files contains:
size, checksum, filename
file name | abrev | use for Migration | comment |
|---|---|---|---|
level1backup-completefiles.txt | min | No | All Level1 files |
level1backup-fullfiles.txt | full | YES | subset of the previous list |
level1backupfiles.txt | complete | No | subset of the previous list |
The total number of files and accumulated file size in these three lists are:
Currently all files (including recon) from the full-listing are backed up to HPSS.
Show the detail of all the file types, sizes in the three lists
The current HPSS tape usage for data backed up from the xrootd cluster to HPSS is: