Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Depending on your data access you may need to submit jobs to a specific farm. This is accomplished by submitting to the appropriate LSF batch queue. Refer to the table below. Jobs for the running experiment should be submitted to high priority queues psnehq and psfehq. Multi-core OpenMPI jobs should be run in either the psnehmpiq or psfehmpiq batch queue, see the following section on "Submitting OpenMPI Batch Jobs". Simulation jobs should be submitted to the low priority queues psfehidle and psfehidle. CPU intensive jobs which don't demand high data throughout should be submitted to the psanacsq queue.

Experimental Hall

Queue

Nodes

Data

Comments

NEH

psnehq

psana11xx

ana01, ana02

Jobs <= 6 cores

 

psnehmpiq

psana11xx,psana12xx

ana01, ana02

OpenMPI jobs > 6 cores, preemptable

 

psnehidle

psana12xx

 

Simulations, preemptable, low priority

FEH

psfehq

psana13xx

ana11, ana12

Jobs <= 6 cores

 

psfehmpiq

psana13xx,psana14xx

ana11, ana12

OpenMPI jobs > 6 cores, preemptable

 

psfehidle

psana14xx

 

Simulations, preemptable, low priority

NEH/FEH

psanacsq

psanacs0xx

ana01, ana02, ana11, ana12

CPU intensive, limited data throughput

Submitting Batch Jobs

LSF (Load Sharing Facility) is the job scheduler used at SLAC to execute user batch jobs on the various batch farms. LSF commands can be run from a number of SLAC servers, but best to use psexport or pslogin. Login first to {{pslogin}} (from SLAC) or to {{psexport}} (from anywhere). From there you can submit a job with the following command:

...

This will submit a job (my_program) to the queue queue psnehq and and write its output to a file named ~/output/job.out.  You You may check on the status of your jobs using the the bjobs command command.

Resource requirements Resource requirements can be specified using the "-R" option.  For For example, to make sure that a job is run on a node with 1 GB (or more) of available memory, use the following: 

No Format
bsub -q psnehq -R "rusagemem=1024" my_program

...