Page History

...

source /reg/g/psdm/etc/ana_env.csh    (for tcsh/csh) OR...
source /reg/g/psdm/etc/ana_env.psconda.sh     (forrequires bash shell)

Starting with ana-0.18.0 (release April 28 2016), the The recommended way to submit MPI batch jobs is

bsub -q psanaq -n 32 24 -o ~/output/%J.out mpirun ~/bin/hello

as opposed to:

bsub -q psanaq -a mympi -n 32 -o ~/output/%J.out ~/bin/hello

This will submit an OpenMPI job  requesting 32 24 processors (-n 3224) to the psanaq batch queue (-q psanaq). The environment variable SIT_RELEASE shows you the ana release number.

...

bsub -q psanaq -n 1612 -R "span[ptile=1]" -o ~/output/%J.out mpirun ~/bin/hello       # for ana-0.18 and greater
bsub -q psanaq -a mympi -n 16 -R "span[ptile=1]" -o ~/output/%J.out ~/bin/hello     # prior to ana-0.18

Will submit an OpenMPI job requesting 16 12 processors (-n 1612) spanned as one processor per host (-R "span[ptile=1]") to the psanaq batch queue (-q psanaq).

bsub -q psanaq -n 12 -R "span[hosts=1]" -o ~/output/%J.out mpirun ~/bin/hello
bsub -q psanaq -a mympi -n 12 -R "span[hosts=1]" -o ~/output/%J.out ~/bin/hello

Will submit an OpenMPI job requesting 12 processors (-n 12) spanned all on one host (-R "span[hosts=1]") to the psanaq batch queue (-q psanaq).

bsub -m "psana1503 psana1509" -q psnehq -n 12 -o ~/output/%J.out mpirun ~/bin/hello
bsub -m "psana1503 psana1509" -q psnehq -a mympi -n 12 -o ~/output/%J.out ~/bin/hello

Will submit an OpenMPI job requesting 12 processors (-n 12) on two nodes (psana1503 and psana1509) to the psnehq batch queue (-q psnehq).

...

bhosts -w ps11farm ps12farm

See list of recently completed ("done") jobs, typically the last 12 hours:

bjobs -d

Getting A High-Priority Interactive Session (When You Have Beam)

...