Glast/xrootd meeting with SCCS
Present: Tony, Tom, Dan, Wilko, Bryson, Bill Weeks, Andy Hanushevsky,
General notes
- GLAST will have its own set of tapes.
- Files can span tapes, so can be larger than 200GB
- Optimal file size would be 1GB or larger
- Writing files to directly to xrootd is possible
- Small chance that file could fail to be overwritten if written multiple times and one server is down
Overhead for getting files from tape
- 20 seconds/mount
- 45 seconds average file seek time (0-90 depending on location on tape)
- tape->disk Max 30Mbytes/sec (in practice more like 10MB)
Tape size 200GB uncompressed, unlikely to get much compression for root files
Babar HPSS usage
- Currently have only 3,000,000 files in HPSS
- Babar files are typically 512MB -> 1GB
- AStore current has 2 800GB file systems (these are not used when xrootd files are staged to/from tape)
Controlling staging to/from xrootd
- xrootd provides a script for archiving files to tape
- Uses lock file for each xrootd file to mark if it has been already staged
- Some control over archiving policy by modifying script.
- xrootd has hook (script) which is called to find files in HPSS
- This hook can be modified to control restore system
- May be possible to set threshold for file size for files to be staged to directly to HPSS
- For files below this size could tar multiple files together before staging to HPSS
- Would need database to enable location of files within tarred files
- Perhaps Navid's existing archiving system could be used for this
Decisions/Action Items
Will set up the following "families" of tapes in HPSS
- glast
- glast/mc
- glast/flight
Will use glastdat account as owner of all archived files.