The FFB system is designed to provide dedicated analysis capabilities during the experiment. After one week from the end of the experiment, the data files will be deleted from FFB and will be available only on one of the offline systems (psana, SDF, or NERSC).
Note 1: The FFB currently offers the fastest file system (WekaIO on NVME disks via IB HDR100) of all LCLS storage systems, but it's limited in size to just a bit more than 400TB: LCLS maybe be forced to remove data from FFB before the nominal week if the running experiments are particularly data intensive.
Note 2: The raw data are copied to the offline storage system and to tape immediately, i.e. in quasi real time during the experiment, not after they have been deleted from FFB. The users generated data, e.g. the scratch and results folders, are copied to the offline system when they are deleted from FFB.
Note 3: For the time being, the new FFB system will be available only for NEH experiments. FEH experiments will still rely on psana resources.
You can access the FFB system from pslogin, psdev or psnx with:
ssh psffb
The experiment data will be available under:
/cds/data/drpsrcf/<instrument>/<experiment>
The experiment folders' names are the same as what you would expect in the offline and are described in the data retention policy.
You can submit your fast feedback analysis jobs to one of the queues shown in the following table. The goal is to assign dedicated resources to up to three experiments for each shift. Please contact your POC to get one of the high priority queues, 1, 2 or 3, assigned to your experiment.
Queue | Comments | Throughput | Nodes | Cores/ | RAM [GB/node] | Default |
|---|---|---|---|---|---|---|
| anaq | For the week after the experiment | 100 | 82 | 61 | 128 | 12hrs |
| ffbl1q | Off-shift queue for experiment 1 | 100 | 8 | 61 | 128 | 12hrs |
| ffbl2q | Off-shift queue for experiment 2 | 100 | 18 | 61 | 128 | 12hrs |
ffbl3q | Off-shift queue for experiment 3 | 100 | 56 | 61 | 128 | 12hrs |
ffbh1q | On-shift queue for experiment 1 | 100 | 8 | 61 | 128 | 12hrs |
ffbh2q | On-shift queue for experiment 2 | 100 | 18 | 61 | 128 | 12hrs |
ffbh3q | On-shift queue for experiment 3 | 100 | 56 | 61 | 128 | 12hrs |
Note that jobs submitted to ffbl<n>q will preempt jobs submitted to anaq and jobs submitted to ffbh<n>q will preempt jobs submitted to ffbl<n>q and anaq. Jobs that are preempted to make resources available to higher priority queues are paused and then are automatically resumed when resources become available.
The FFB system uses SLURM for submitting jobs - information on using SLURM can be found on the Submitting SLURM Batch Jobs page.
Overview
Content Tools