Content

Cloning packages from git

git clone git@github.com:slac-lcls/lcls2.git

git clone git@github.com:lcls-psana/Detector.git

git clone git@github.com:lcls-psana/PSCalib.git

git clone git@github.com:lcls-psana/CalibManager.git

git clone git@github.com:lcls-psana/pyimgalgos.git

git clone git@github.com:lcls-psana/.git

git clone git@github.com:lcls-psana/SConsTools.git

git clone git@github.com:lcls-psana/pdsdata.git
git clone git@github.com:lcls-psana/psalg.git

git clone git@github.com:lcls-psana/data_test.git /tmp/data_test

git clone git@github.com:slaclab/anarel-manage.git manage

git clone git@github.com:slac-lcls/psana1-feedstock.git

git clone https://github.com/slac-lcls/psana1-feedstock

PCDS

LCLS1

Development

cd mytestrel

rm .sit_conda_env

source /cds/sw/ds/ana/conda1/manage/bin/psconda.sh

# If a different environment needs to be activated, it needs to be activated here
# before setup_testrel. see New Style Test Releases

source /cds/sw/ds/ana/conda1/manage/bin/setup_testrel

###  defaults to py3, optionally add a "-py2" flag for py2

source /cds/sw/ds/ana/conda1/manage/bin/psconda.sh -py2

source /cds/sw/ds/ana/conda1/manage/bin/setup_testrel

source /cds/sw/ds/ana/conda1/manage/bin/conda_activate ana-4.0.37-py3

git clone git@github.com:lcls-psana/Detector.git

scons

clean_lcls1_env # from ~/bin/clean_lcls1_env

External Packages

If working on external packages (for example, ndarray, psalg, pdsdata, python), you need to also to check out the corresponding proxy package. For example: ndarray_ext, Python, psalg_ext, pdsdata_ext, etc. The Python proxy package allows the python header files to be visible to C++ code.

Production

source /cds/sw/ds/ana/conda1/manage/bin/psconda.sh -py2

   defaults to py3, optionally add a "-py2" flag for py2

List release package tags

For example ana-4.0.45 tags can be seen in

/cds/sw/ds/ana/conda1/inst/envs/ana-4.0.45/data/anarelinfo/psana-conda-tags

Change test release

List available release by command

conde env list

Then set desired release ana-4.0.## with script/command conda_activate.

Sequence of commands mast be:

source /cds/sw/ds/ana/conda1/manage/bin/psconda.sh

source conda_activate ana-4.0.##

source /cds/sw/ds/ana/conda1/manage/bin/setup_testrel

LCLS2

Development

cd mytestrel

source /cds/sw/ds/ana/conda2/manage/bin/psconda.sh

git clone git@github.com:slac-lcls/lcls2.git

cd lcls2

.  setup_env.sh

./clean_lcls2.sh

./build_all.sh

Test build production

./clean_lcls2.sh

./build_all.sh -p install

but keep in mind that any changes to the python will require that command to be issued again ("non-develop" mode).

Then test commands etc. and return back to development

./clean_lcls2.sh

./build_all.sh

Change test release

List available release by command

conde env list

Then set desired release ps-4.5.## with commands

conda deactivate

conda activate ps-4.5.##

Production

source /cds/sw/ds/ana/conda2/manage/bin/psconda.sh

Test release for users of UED

source /reg/g/pcds/dist/pds/ued/scripts/setup_env.sh

Build ami2

ssh -Y psbuild-rhel7

git clone https://github.com/slac-lcls/lcls2.git
git clone https://github.com/slac-lcls/ami.git
cd lcls2
source setup_env.sh
./build_all.sh
cd ../ami
./build_all.sh

ami-local -b 1 -f interval=1 psana://exp=tmoc00118,run=222,dir=/cds/data/psdm/prj/public01/xtc

ami Simple Introductory Examples

S3DF

Running at S3DF

lcls1:

source /sdf/group/lcls/ds/ana/sw/conda1/manage/bin/psconda.sh -py2

source /sdf/group/lcls/ds/ana/sw/conda1/manage/bin/psconda.sh

source /sdf/group/lcls/ds/ana/sw/conda1/manage/bin/setup_testrel

lclc2:

unset LD_LIBRARY_PATH
unset PYTHONPATH

# for psana
#source /cds/sw/ds/ana/conda2-v2/inst/etc/profile.d/conda.sh
#export CONDA_ENVS_DIRS=/cds/sw/ds/ana/conda2/inst/envs/

# for s3df
source /sdf/group/lcls/ds/ana/sw/conda2/inst/etc/profile.d/conda.sh
export CONDA_ENVS_DIRS=/sdf/group/lcls/ds/ana/sw/conda2/inst/envs
# Mikhail: root of psdm directories
export DIR_PSDM=/sdf/group/lcls/ds/ana/
export SIT_PSDM_DATA=/sdf/data/lcls/ds/

conda activate ps-4.5.26

RELDIR="$( cd "$( dirname $(readlink -f "${BASH_SOURCE[0]}") )" && pwd )"
export PATH=$RELDIR/install/bin:${PATH}
pyver=$(python -c "import sys; print(str(sys.version_info.major)+'.'+str(sys.version_info.minor))")
export PYTHONPATH=$RELDIR/install/lib/python$pyver/site-packages
# for procmgr
export TESTRELDIR=$RELDIR/install
export PROCMGR_EXPORT=RDMAV_FORK_SAFE=1,RDMAV_HUGEPAGES_SAFE=1  # See fi_verbs man page regarding fork()

# cpo: seems that in more recent versions blas is creating many threads
export OPENBLAS_NUM_THREADS=1
# cpo: getting intermittent file-locking issue on ffb, so try this
export HDF5_USE_FILE_LOCKING=FALSE
# for libfabric. decreases performance a little, but allows forking
export RDMAV_FORK_SAFE=1
export RDMAV_HUGEPAGES_SAFE=1
# Mikhail: root of psdm directories
#export DIR_PSDM=/cds/group/psdm

# cpo: workaround a qt bug which may no longer be there (dec 5, 2022)
if [ ! -d /usr/share/X11/xkb ]; then
    export QT_XKB_CONFIG_ROOT=${CONDA_PREFIX}/lib
fi

# needed by Ric to get correct libfabric man pages
export MANPATH=$CONDA_PREFIX/share/man${MANPATH:+:${MANPATH}}

for developers: source lcls2/setup_env.sh

for users: source /sdf/group/lcls/ds/ana/sw/conda2/manage/bin/psconda.sh

on psana: export SIT_DATA=/cds/group/psdm/data

export SIT_DATA=/sdf/group/lcls/ds/ana/data
Ex:
  sdfdtn001%  detnames /sdf/data/lcls/ds/xpp/xpptut15/xtc/e665-r0240-s02-c00.xtc

/sdf/group/lcls/ds/ana/

data, detector, sw

FFB

lclc2:

to access /cds/data/drpsrcf/tst/tstx00117/xtc

It’s “ssh psffb”.  add the “dir=“ option to DataSource, or “export SIT_PSDM_DATA=/cds/data/drpsrcf”

SLURM commands

sinfo -N -r -l
squeue -u $USER | grep 197 | awk '{print $1}' | xargs -n 1 scancel
squeue -u <username>

scancel <job_id>

scontrol show partition <partition>
scontrol show node <hostname>
scontrol show job <job_id>

scontrol hold <job_list>
scontrol release <job_list>
scontrol show hostnames

sinfo --states=idle

srun --partition milano --nodelist=sdfmilan112 --account lcls:prjdat21 -n 1 --time=02:00:00 --exclusive --pty /bin/bash

srun --partition milano --account lcls:prjdat21 -n 1 --time=02:00:00 --exclusive --pty /bin/bash

srun --partition milano --reservation=lcls:scaling --account lcls:prjdat21 -n 1 --time=02:00:00 --exclusive --pty /bin/bash

Patching release

2023-09-15 O'Grady, Paul Christopher <cpo@slac.stanford.edu> To: Dubrovin, Mikhail

For the log the patch commands are below:

(ana-4.0.51-py3) pwd
/sdf/group/lcls/ds/ana/sw/conda1/inst/envs/ana-4.0.51-py3/lib/python3.9/site-packages/Detector
(ana-4.0.51-py3) cp ~cpo/git/Detector/src/AreaDetector.py AreaDetector.py
(ana-4.0.51-py3) cd ../PSCalib/
(ana-4.0.51-py3) cp ~cpo/git/PSCalib/src/GlobalUtils.py GlobalUtils.py
(ana-4.0.51-py3)

chris

References

• New Style Test Releases
• Merge as we go and py3 local release - DEPRECATED (deprecated)
• Running at S3DF