Page History

...

• Intend to use xrootd for storing data files (mainly *.root, *.fits)
  • MC simulations, test data and real data from satellite
  • Some files will be relatively small (10's of MBytes, so Wilko is developing system to tar these files after they are put into xrootd, but before they are put on tape).
• Want xrootd to manage many physical disks
• Want xrootd to manage archiving of files to HPSS (with recovery from tape expected to be rare)
• All files to be put into xrootd should also be registered in data catalog
  • Files are first put into xrootd, then registered in data catalog
    • This is not an atomic transaction
  • Data catalog has "crawler" which verifies existence of files after they have been added
    • Extracts checksum, creation date, # events etc
    • If file exists in multiple locations (e.g. xrootd and NFS) checks consistency of files.

...

1. It would be nice if xrootd kept the original date when a file is copied in, and maintained this even if added to/restored from HPSS.
2. The command syntax, error messages, and returncode for commonly used xrootd commands could be
   improved to be easier to use within (python) scripts.
3. lack of a reasonable way to list files in xrootd (e.g., "ls" command)
   1. A possible alternative solution would be a nightly job which lists files on all servers, combines them together, and flags inconsistencies (e.g. same file with different size/date).
4. Ability to migrate (or duplicate) data between disks without going via HPSS.

To Do List

1. Complete implementation of tar/archive/retrieval system