Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Table of Contents

Batch Nodes/Queues

Depending on your data access you may need to submit jobs to a specific farm. This is accomplished by submitting to the appropriate LSF batch queue. Refer to the table below for the batch resources available in psana. Jobs for the current experiment should be submitted to the high priority queues psnehhiprioq and psfehhiprioq running against the Fast Feedback storage layer (FFB) located at /reg/d/ffb/<hutch>/<experiment> as shown HERE. Jobs for the off-shift experiment should be submitted to psnehprioq and psfehprioq. Only psneh(hi)prioq/psfeh(hi)prioq should access the FFB.  When in doubt, use psanaq.in the Fast Feedback System which allocates resources for the most recent experiments and is significantly faster and more modern than psana.

Submit your job from an interactive node (where you land after doing ssh psana). LSF will run the submitted job on the specified queue using nodes listed in the table below. All nodes in the queues listed below run rhel7RHEL7. By submitting from an interactive node (also running rhel7RHEL7), you will ensure that your job inherits a rhel7 RHEL7 environment.

...


Location

Queue

Nodes

Data

Comments

Throughput

(

[Gbit/s

)

]

Cores
NodesCores/Node

RAM

(

[GB/node

)

Default
Time
Limit 

Building 50

]

Time limit 

psanaqpsana11xx, psana12xx,psana13xx, psana14xx
ALL (no FFB)
Primary psana queue40
960
80122448hrs
 
psdebugqsame as psanaq
same as psanaq
SHORT DEBUGGING ONLY (preempts psanaq jobs)40
24
80122410min
 
psanaidleqpsana11xx, psana12xx,psana13xx, psana14xx

 

Jobs preemptable by psanaq40
960
80122448hrs

NEH

psnehhiprioq

psanamq

psana15xx

FFB for AMO, SXR, XPP

Current NEH experiment on FFB ONLY

, psana16xx

Larger memory nodes

40
288
401612824hrs
 

psnehprioq

psana15xx

FFB for AMO, SXR, XPP

Off-shift NEH experiment on FFB ONLY

402881612824hrs

 

psnehq

psana15xx

 

Jobs preemptable by psneh(hi)prioq

10
psanagpuqpsanagpu106-psanagpu118GPU nodes1013
288
16128
48hrs

FEH

psfehhiprioq

psana16xx

FFB for XCS, CXI, MEC

Current FEH experiment on FFB ONLY

4028816128
24hrs
 

psfehprioq

psana16xx

FFB for XCS, CXI, MEC

Off-shift FEH experiment on FFB ONLY

402881612824hrs

 

psfehq

psana16xx

 

Jobs preemptable by psfeh(hi)prioq

102881612848hrs

Submitting Jobs

Information on submitting jobs to the SLURM system at LCLS can be found on this page: Submitting SLURM Batch Jobs

...