Agenda

• #Requirements for controlling how files are archived
• Archiving small files
  • htar vs tar
  • db vs file
  • mpsstage script
    • how to modify for accessing files from archives
    • how to handle simultaneous requests for different from from the same archive
• File access control (nfs group vs file)
• Plan of attack and action items

Requirements for controlling how files are archived

• Need to be able to control how files are handled based on folder and file type
  • Perhaps via regular expression – although Wilko is worried about performance
• Need to be able to specify:
  • File should never be archived
  • File should be archived immediately
  • File should be archived once it is a certain age
• Need to be able specify minimum size for non-tar archiving
• Need to be able to specify disposition of disk file after archived
  • Delete immediately
  • Never delete
  • Delete once it is a certain age

Somehow we need a mechanism for modifying these rules fairly easily, for example when a MC task is running the default maybe not to archive it, but once the files have been verified then we may want to immediately archive them, and perhaps delete some from disk.

Archive configuration file

Strawman proposal