Page History

...

1. scratch folder is made non accessible to users.
2. files and directories below the ffb scratch/ are moved to the scratch/ffb/ on the offline filesystem: 
       /cds/data/psdm/<instr>/<expt>/scratch/ffb/ 
   except for hdf5 files in the smalldata folder (see next).
   Once the data are on the offline scratch the Data Retention Policy applies. The transfer preserves the files mtime which is used by the cleanup.
3. hdf5 files below scratch/hdf5/smalldata/  are moved the the hdf5/smalldata/ folder on the offline filesystem, e.g.
   /cds/data/drpsrcf/mfx/mfx123456/scratch/smalldata/*.h5  -> /cds/data/psdm/psdm/mfx/mfx123456/hdf5/smalldata/
   1. Only h5 files (and directories) below scratch/hdf5/smalldata/ are copied to the hdf5/smalldata folder on the offline storage.
   2. Files that don't match the rule in a. will be moved to the offline scratch/ffb/hdf5 folder.
   3. If a h5 file already exists on the offline storage and is newer than the one on the ffb the ffb file will not be copied but just removed.

...