Page History

In 2021 LCLS switch to the SLURM batch system.

Information on submitting jobs to the SLURM system at LCLS can be found on this page: Submitting SLURM Batch Jobs

Information on the Automatic Run Processing system (ARP) can be found on this page: Automatic Run Processing (ARP).  This is also usable at sites like NERSC and SDF.

A "cheat sheet" showing similar commands on LSF and SLURM can be found here: https://slurm.schedmd.com/rosetta.pdf

Refer to the table below for the batch resources available in psana.

Table of Contents

Batch Nodes/Queues

Depending on your data access you may need to submit jobs to a specific farm. This is accomplished by submitting to the appropriate LSF batch queue. Refer to the table below. Jobs for the current experiment should be submitted to the high priority queues psnehhiprioq and psfehhiprioq running against the Fast Feedback storage layer (FFB) located at /reg/d/ffb/<hutch>/<experiment> as shown HERE. Jobs for the off-shift experiment should be submitted to psnehprioq and psfehprioq. Only psneh(hi)prioq/psfeh(hi)prioq should access the FFB.  When in doubt, use psanaq.

Submit your job from an interactive node (where you land after doing ssh psana). LSF will run the submitted job on the specified queue using nodes listed in the table below. All nodes in the queues listed below run rhel7RHEL7. By submitting from an interactive node (, also running rhel7)RHEL7, you will ensure that your job inherits a rhel7 environment.

RHEL7 environment.

Note 1: Jobs for the current experiment can be submitted to fast feedback (FFB) queues, which allocate resources for the most recent experiments. The FFB queues in the tables below are for LCLS-II experiments (TMO, RIX and UED). The FEH experiments (LCLS-I, including XPP) can submit FFB jobs to the new Fast Feedback System.

Submitting Jobs

Information on submitting jobs to the SLURM system at LCLS can be found on this page: Submitting SLURM Batch Jobs

Information on the submitting jobs to the (deprecated) LSF batch system can be found on this page: Submitting LSF Batch Jobs

Information on the Automatic Run Processing system (ARP) can be found on this page: Automatic Run Processing.  This is also usable at sites like NERSC and SDF.

...

...