...

Note 2: The raw data are copied to the offline storage system and to tape immediately, i.e. in quasi real time during the experiment, not after they have been deleted from FFB. The users generated data created in the scratch/ folder are copied moved to the offline system when they are the experiment is deleted from FFB. The content of the scratch folder will be moved to the scratch/ffb/ sub-folder on the ana file systemsthe FFB described in Lifetime of data on the FFB.

Note 3: For the time being, the new FFB system will be available only for FEH experiments. NEH experiments will still rely on psana resources.

You can access the FFB system from pslogin, psdev or psnx with:

% ssh psffb

The experiment data will be available under:

...

export SIT_PSDM_DATA=/cds/data/drpsrcf

The experiment folders' folder names are the same as what you would expect in the offline and are described in the data retention policy.

...

• the umask is applied when creating files and directories which violates the  ACL specs. As the default umask is 022 the group write permission will be removed. We recommend to set ones umask to:

% umask 0002

Anchor
cleanup cleanup

Lifetime of data on the FFB

xtc folder

• xtc files are immediately copied to the offline filesystem
• the lifetime on the ffb is dictated by how much data is generated
  • typically files stay on the ffb during the run-time of an experiment
  • however if space is need files from previous shifts might get purged
  • after an experiment is done the ffb should not be used anymore except if discussed with the POC

scratch folder

Once an experiment has been complete and all xtc files have been removed the ffb scratch folder is moved to the experiments scratch folder on the ana-filesystems. The following rules are applied:

1. scratch folder is made non accessible by the experiment
2. files and directories below the ffb scratch/ are moved to the scratch/ffb/ on the offline filesystem:  /reg/d/psdm/<instr>/<expt>/scratch/ffb/  except for hdf5 files in the smalldata folder (see next).
3. hdf5 files below scratch/hdf5/smalldata/  are moved the the hdf5/smalldata/ folder on the ana-filesystemsoffline filesystem, e.g.
   /cds/data/drpsrcf/mfx/mfx123456/scratch/smalldata/*/*hdf5  .h5  -> /regcds/data/dpsdm/psdm/mfx/mfx123456/hdf5/smalldataall other files/directories below scratch /
   (only .h5 files are moved to /reg/d/psdm/<instr>/<expt>/the hdf5/smalldata other files will be moved below scratch/ffb/

Access to Lustre ana-filesystems

...