Page History

...

Experiment	Run	Comment
tmoc00118	222	Generic TMO dark data
tmoacr019	4,5,6	xtcav dark,lasing-off,lasing-on (cpo thinks, not certain)
rixx43518	34	A DAQ "fly scan" of motor (see ami#FlyScan:MeanVs.ScanValue)
rixx43518	45	A DAQ "step scan" of two motors
rixl1013320	63	Rix stepping delay scan of both Vitara delay and ATM delay stage (lxt_ttc scan) at a single mono energy
rixl1013320	93	Rix continuous mono scan with laser on/off data at a single time delay
rixx1003821	55	An infinite sequence with two slow andors running at different rates
rixx1003821	68	A finite burst sequence with one andor
uedcom103	7	epix10ka data
ueddaq02	569	epix10ka data

...

Code Block

language	bash
title	submit_large_psana2.sh

#!/bin/bash
#SBATCH --partition=milano
#SBATCH --job-name=run_large_psana2
#SBATCH --output=output-%j.txt
#SBATCH --error=output-%j.txt
#SBATCH --nodes=3
#SBATCH --exclusive
#SBATCH --time=10:00

# Configure psana2 parallelization
source setup_hosts_openmpi.sh

# Run your job with #ranks <= (#nodes - 1) * 120 + 1 or use $PS_N_RANKS 
mpirun -np $PS_N_RANKS --hostfile $PS_HOST_FILE python test_mpi.py

...

Code Block

language	bash
title	setup_hosts_openmpi.sh

############################################################
# First node must be exclusive to smd0
# * For openmpi, slots=1 must be assigned to the first node.
############################################################

# Get list of hosts by expand shorthand node list into a 
# line-by-line node list
host_list=$(scontrol show hostnames $SLURM_JOB_NODELIST)
hosts=($host_list)

# Write out to host file by putting rank 0 on the first node
host_file="slurm_host_${SLURM_JOB_ID}"
for i in "${!hosts[@]}"; do
    if [[ "$i" == "0" ]]; then
        echo ${hosts[$i]} slots=1 > $host_file
    else
        echo ${hosts[$i]} >> $host_file
    fi
done

# Export hostfile for mpirun  
export PS_HOST_FILE=$host_file

# Calculate no. of ranks available in the job. 
export PS_N_RANKS=$(( SLURM_CPUS_ON_NODE * ( SLURM_JOB_NUM_NODES - 1 ) + 1 ))

...

Page tree

Versions Compared

Old Version 148

New Version Current

Key