Glast/xrootd meeting with SCCS

Present: Tony, Tom, Dan, Wilko, Bryson, Bill Weeks, Andy Hanushevsky,

General notes

  • GLAST will have its own set of tapes.
  • Files can span tapes, so can be larger than 200GB
  • Optimal file size would be 1GB or larger
  • Writing files to directly to xrootd is possible
    • Small chance that file could fail to be overwritten if written multiple times and one server is down

Overhead for getting files from tape

  • 20 seconds/mount
  • 45 seconds average file seek time (0-90 depending on location on tape)
  • tape->disk Max 30Mbytes/sec (in practice more like 10MB)

Tape size 200GB uncompressed, unlikely to get much compression for root files

Babar HPSS usage

  • Currently have only 3,000,000 files in HPSS
  • Babar files are typically 512MB -> 1GB
  • AStore current has 2 800GB file systems (these are not used when xrootd files are staged to/from tape)

Controlling staging to/from xrootd

  • xrootd provides a script for archiving files to tape
    • Uses lock file for each xrootd file to mark if it has been already staged
    • Some control over archiving policy by modifying script.
  • xrootd has hook (script) which is called to find files in HPSS
    • This hook can be modified to control restore system
    • May be possible to set threshold for file size for files to be staged to directly to HPSS
    • For files below this size could tar multiple files together before staging to HPSS
      • Would need database to enable location of files within tarred files
      • Perhaps Navid's existing archiving system could be used for this

Decisions/Action Items

Will set up the following "families" of tapes in HPSS

  • glast
  • glast/mc
  • glast/flight

Will use glastdat account as owner of all archived files.

  • No labels