Notes about building psana releases.
psana1
Assemble all packages source code in tar.gz file
pslogin
ssh psbuild-rhel7-01 -l psreldev
Set environment:
source /cds/sw/ds/ana/conda1/inst/etc/profile.d/conda.sh
conda activate conda_build
cd /cds/sw/ds/ana/conda1/manage
look at subdirectory scratch/ to see what is the latest release, e.g. psana-conda-4.0.55, then create next:
bin/ana-rel-admin --force --cmd psana-conda-src --name 4.0.56 --basedir `pwd` --tagsfile psana-conda-svn-pkgs[-test]
Algorithm of ana-rel-admin
Command bin/ana-rel-admin ... assembles all the source code from github and svn archives with the latest tags into a .tar.gz file
--basedir`pwd` is converted to /cds/sw/ds/ana/conda1/manage/
input: < config/psana-conda-svn-pkgs # contains the list of packages with source code
output:
> scratch/psana-conda-4.0.56/<packages> # clones of all packages form git or svn
> git clone git@github.com:lcls-psana/data_test.git /tmp/data_test
> scratch/psana-conda-4.0.56/psana-conda-tag # tags extended with version like 'tag': 'V02-00-76'
> scratch/psana-conda-4.0.56/psana-conda-4.0.56/.sit_release # contains psana-conda-4.0.56
> downloads/anarel/psana-conda-4.0.56.tar.gz # archived content of scratch/psana-conda-4.0.56
> dumps at the end sha256sum .../psana-conda-4.0.56.tar.gz > 443440bf7a7fdbcdf89314b68c07f97f5a89dff38d6da98685e74f0671fac37d
Command bin/ana-rel-admin creates file accessible through url:
file:///cds/sw/ds/ana/conda1/manage/downloads/anarel/psana-conda-4.0.56.tar.gz
sha256sum /cds/sw/ds/ana/conda1/manage/downloads/anarel/psana-conda-4.0.56.tar.gz
> 443440bf7a7fdbcdf89314b68c07f97f5a89dff38d6da98685e74f0671fac37d
Conda build
ssh pslogin (not as psreldev yet...)
ssh psbuild-rhel7-01 -l psreldev
cd git/psana1-feedstock/
Update in ~psreldev/git/psana1-feedstock/recipe/meta.yaml fields for set version and sha256:
{% set version = '4.0.56' %}
...
sha256: 443440bf7a7fdbcdf89314b68c07f97f5a89dff38d6da98685e74f0671fac37d
...
Set environment:
source /cds/sw/ds/ana/conda1/inst/etc/profile.d/conda.sh # OR conda deactivate after previous conda_build
conda activate conda_build_py3
Execute command:
conda build -c lcls-i -c conda-forge recipe
> /cds/home/p/psreldev/conda-bld/linux-64/psana-4.0.56-py39hb869b97_1.tar.bz2
xorg-xf86vidmodeproto: 2.3.1-h7f98852_1002 conda-forge
xorg-xproto: 7.0.31-h7f98852_1007 conda-forge
xz: 5.2.6-h166bdaf_0 conda-forge
zeromq: 4.3.5-h59595ed_0 conda-forge
zipp: 3.17.0-pyhd8ed1ab_0 conda-forge
zlib: 1.2.13-hd590300_5 conda-forge
zstd: 1.5.5-hfc55251_0 conda-forge
Preparing transaction: ...working... done
Verifying transaction: ...working... done
Executing transaction: ...working... done
export PREFIX=/cds/home/p/psreldev/conda-bld/psana_1700187012881/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl
export SRC_DIR=/cds/home/p/psreldev/conda-bld/psana_1700187012881/test_tmp
+ psana -h
Usage: psana [options] [dataset ...]
Available options:
{-h|-?|--help } print help message
{-v|--verbose } (incr) verbose output, multiple allowed (initial: 0)
{-q|--quiet } (incr) quieter output, multiple allowed (initial: 2)
{-b|--calib-dir } path calibration directory name, may include {exp} and {instr}, if left empty then do not do calibrations (default: "")
{-c|--config } path configuration file, by default use psana.cfg if it exists (default: "")
{-e|--experiment } string experiment name, format: XPP:xpp12311 or xpp12311, by default guess it from data (default: "")
{-j|--job-name } string job name, default is to generate from input file names (default: "")
{-m|--module } name module name, more than one possible
{-n|--num-events } number maximum number of events to process, 0 means all (default: 0)
{-s|--skip-events} number number of events to skip (default: 0)
{-p|--num-cpu } number number greater than 0 enables multi-processing (default: 0)
{-o|--option } string configuration options, format: module.option[=value]
Positional parameters:
dataset - input dataset specification (list of file names or exp=cxi12345:run=123:...)
+ exit 0
Resource usage statistics from testing psana:
Process count: 1
CPU time: Sys=0:00:00.1, User=-
Memory: 1.4M
Disk usage: 12B
Time elapsed: 0:00:02.4
TEST END: /cds/home/p/psreldev/conda-bld/linux-64/psana-4.0.56-py39hb869b97_1.tar.bz2
Renaming work directory '/cds/home/p/psreldev/conda-bld/psana_1700187012881/work' to '/cds/home/p/psreldev/conda-bld/psana_1700187012881/work_moved_psana-4.0.56-py39hb869b97_1_linux-64_main_build_loop'
INFO:conda_build.utils:Renaming work directory '/cds/home/p/psreldev/conda-bld/psana_1700187012881/work' to '/cds/home/p/psreldev/conda-bld/psana_1700187012881/work_moved_psana-4.0.56-py39hb869b97_1_linux-64_main_build_loop'
INFO conda_build.utils:shutil_move_more_retrying(2075): Renaming work directory '/cds/home/p/psreldev/conda-bld/psana_1700187012881/work' to '/cds/home/p/psreldev/conda-bld/psana_1700187012881/work_moved_psana-4.0.56-py39hb869b97_1_linux-64_main_build_loop'
shutil.move(work)=/cds/home/p/psreldev/conda-bld/psana_1700187012881/work, dest=/cds/home/p/psreldev/conda-bld/psana_1700187012881/work_moved_psana-4.0.56-py39hb869b97_1_linux-64_main_build_loop)
INFO:conda_build.utils:shutil.move(work)=/cds/home/p/psreldev/conda-bld/psana_1700187012881/work, dest=/cds/home/p/psreldev/conda-bld/psana_1700187012881/work_moved_psana-4.0.56-py39hb869b97_1_linux-64_main_build_loop)
INFO conda_build.utils:shutil_move_more_retrying(2082): shutil.move(work)=/cds/home/p/psreldev/conda-bld/psana_1700187012881/work, dest=/cds/home/p/psreldev/conda-bld/psana_1700187012881/work_moved_psana-4.0.56-py39hb869b97_1_linux-64_main_build_loop)
# Automatic uploading is disabled
# If you want to upload package(s) to anaconda.org later, type:
anaconda upload /cds/home/p/psreldev/conda-bld/linux-64/psana-4.0.56-py39hb869b97_1.tar.bz2
# To have conda build upload to anaconda.org automatically, use
# $ conda config --set anaconda_upload yes
anaconda_upload is not set. Not uploading wheels: []
####################################################################################
Resource usage summary:
Total time: 1:22:33.6
CPU usage: sys=0:03:04.5, user=0:27:14.5
Maximum memory usage observed: 1.6G
Total disk usage observed (not including envs): 285.6M
####################################################################################
Source and build intermediates have been left in /cds/home/p/psreldev/conda-bld.
There are currently 1 accumulated.
To remove them, you can run the ```conda build purge``` command
(conda_build) [psreldev@psbuild-rhel7-01 psana1-feedstock]$
Upload the file to anaconda lcls-i channel
anaconda upload -u lcls-i /sdf/group/lcls/ds/ana/sw/conda1/inst/envs/conda_build2/conda-bld/linux-64/psana-4.0.57-py39hed0727e_1.tar.bz2
(conda_build2) [psreldev@sdfiana001 psana1-feedstock]$ pwd
/sdf/home/p/psreldev/git/psana1-feedstock
(conda_build2) [psreldev@sdfiana001 psana1-feedstock]$ anaconda upload -u lcls-i /sdf/group/lcls/ds/ana/sw/conda1/inst/envs/conda_build2/conda-bld/linux-64/psana-4.0.57-py39hed0727e_1.tar.bz2
Using Anaconda API: https://api.anaconda.org
Using "lcls-i" as upload username
Processing "/sdf/group/lcls/ds/ana/sw/conda1/inst/envs/conda_build2/conda-bld/linux-64/psana-4.0.57-py39hed0727e_1.tar.bz2"
Detecting file type...
File type is "Conda"
Extracting conda attributes for upload
Creating package "psana"
Creating release "4.0.57"
The action you are performing requires authentication, please sign in:
Using Anaconda API: https://api.anaconda.org
Username: dubrovin
dubrovin's Password:
login successful
Using Anaconda API: https://api.anaconda.org
Using "lcls-i" as upload username
Processing "/sdf/group/lcls/ds/ana/sw/conda1/inst/envs/conda_build2/conda-bld/linux-64/psana-4.0.57-py39hed0727e_1.tar.bz2"
Detecting file type...
File type is "Conda"
Extracting conda attributes for upload
Creating package "psana"
Creating release "4.0.57"
Uploading file "lcls-i/psana/4.0.57/linux-64/psana-4.0.57-py39hed0727e_1.tar.bz2"
15.4MB [00:01, 12.6MB/s]
Upload complete
conda located at:
https://anaconda.org/lcls-i/psana
(conda_build2) [psreldev@sdfiana001 psana1-feedstock]$
Create new environment
goal: create a real 4.0.57
conda create --name psana-conda-4.0.57 --clone psana-conda-4.0.56
(conda_build2) [psreldev@sdfiana001 psana1-feedstock]$ conda create --name ana-4.0.57-py3 --clone ana-4.0.55-py3
Source: /sdf/group/lcls/ds/ana/sw/conda1/inst/envs/ana-4.0.55-py3
Destination: /sdf/group/lcls/ds/ana/sw/conda1/inst/envs/ana-4.0.57-py3
Packages: 488
Files: 11
Downloading and Extracting Packages:
font-ttf-dejavu-sans | ########################################################################################################################################################################### | 100%^
Downloading and Extracting Packages:
Preparing transaction: done
Verifying transaction: done
Executing transaction: |
For Linux 64, Open MPI is built with CUDA awareness but this support is disabled by default.
To enable it, please set the environment variable OMPI_MCA_opal_cuda_support=true before
launching your MPI processes. Equivalently, you can set the MCA parameter in the command line:
mpiexec --mca opal_cuda_support 1 ...
In addition, the UCX support is also built but disabled by default.
To enable it, first install UCX (conda install -c conda-forge ucx). Then, set the environment
variables OMPI_MCA_pml="ucx" OMPI_MCA_osc="ucx" before launching your MPI processes.
Equivalently, you can set the MCA parameters in the command line:
mpiexec --mca pml ucx --mca osc ucx ...
Note that you might also need to set UCX_MEMTYPE_CACHE=n for CUDA awareness via UCX.
Please consult UCX's documentation for detail.
done
#
# To activate this environment, use
#
# $ conda activate ana-4.0.57-py3
#
# To deactivate an active environment, use
#
# $ conda deactivate
(conda_build2) [psreldev@sdfiana001 psana1-feedstock]$
conda deactivate
conda activate psana-conda-4.0.57
conda install -c lcls-i -c conda-forge psana=4.0.57 # Chris? --experimental-solver=libmamba
have to do this on both old-psana and new-s3df as psreldev
See details in ~/.bash_history or ~/.k5login
cd /cds/sw/ds/ana/conda1/
if ./manage is not available
git clone git@github.com:slaclab/anarel-manage.git manage
cd manage
update it if necessary
git pull --rebase
conda create -n conda_build_py3 python=3.9 anaconda
conda env remove --name conda_build_py3
see /cds/sw/ds/ana/conda1/inst/envs/ OR conda info --envs
ssh psbuild-rhel7-01 -l psreldev
cd git/psana1-feedstock/
sha256sum /cds/sw/ds/ana/conda1/manage/downloads/anarel/psana-conda-4.0.56.tar.gz
> 443440bf7a7fdbcdf89314b68c07f97f5a89dff38d6da98685e74f0671fac37d
~psreldev/git/psana1-feedstock/recipe/meta.yaml
cd git
rm -rf psana1-feedstock/
git clone https://github.com/slac-lcls/psana1-feedstock
Set environment (chris):
source /cds/sw/ds/ana/conda2/manage/bin/psconda.sh. # Chris: conda2 because it needs py3, but conda1 also works with py3...
conda deactivate
conda activate conda_build
OR:
source /cds/sw/ds/ana/conda2/manage/bin/psconda.sh
conda create -n conda_build_py3 python conda-build anaconda-client
conda activate conda_build_py3
ssh psbuild-rhel7-01 -l psreldev
cd git
rm -rf psana1-feedstock/
git clone https://github.com/slac-lcls/psana1-feedstock
cp /cds/sw/ds/ana/conda1/manage/downloads/anarel/psana-conda-4.0.56.tar.gz /reg/g/psdm/web/swdoc/tutorials/
Issue: PermissionError: [Errno 13] Permission denied: '/cds/sw/ds/ana/conda2/inst/envs/conda_build/pkgs/libselinux-cos6-x86_64-2.0.94-h9b0a68f_1105'
Chris: cp ~cpo/.condarc ~/. # if ~/.condarc is not available
Valerio: conda create -n local_conda_build python conda-build conda-validate anaconda-client
scons test-Detector
- git repository data_test_access cloned with all other psana packages
- location for data_test can be controlled in config/psana-conda-svn-pkgs:
- subdir=/tmp - clones this package as /tmp/data_test
- data_test_access is included but data_test is ignored in
tar cfvz /cds/sw/ds/ana/conda1/manage/downloads/anarel/psana-conda-4.0.56.tar.gz psana-conda-4.0.56
Build on s3df
Initialization for build on s3df
IT IS ALREADY DONE ONCE
clone anarel-manage
cd .../con-build OR cd /sdf/group/lcls/ds/ana/sw/conda1/
. /sdf/group/lcls/ds/ana/sw/conda1/manage/bin/psconda.sh
git clone git@github.com:slaclab/anarel-manage.git
make github seen from psana nodes
ema ~/.gitconfig
\[http\] - remove
Run bin/ana-rel-admin on s3df
login as myself
> s3dfnx ends up as dubrovin@sdflogin002
> psana ... because /afs/slac/g/pcds/svn/pdsdata is seen from psana nodes only... (Wilko)
cd <path-to>/anarel-manage
login as psreldev
> s3dfnx ends up as dubrovin@sdflogin002
ssh psana -l psreldev
cd /sdf/group/lcls/ds/ana/sw/conda1/manage
conda activate and run ana-rel-admin
source /sdf/group/lcls/ds/ana/sw/conda1/manage/bin/psconda.sh
conda deactivate
conda activate conda_build
bin/ana-rel-admin --force --cmd psana-conda-src --name 4.0.56 --basedir `pwd`
Command bin/ana-rel-admin creates file
/sdf/group/lcls/ds/ana/sw/conda1/manage/downloads/anarel/psana-conda-4.0.56.tar.gz
accessible in ~psreldev/git/psana1-feedstock/recipe/meta.yaml through url:
file:///sdf/group/lcls/ds/ana/sw/conda1/manage/downloads/anarel/psana-conda-4.0.56.tar.gz
conda1-v3
Valerio generated new conda1-v3
source /sdf/group/lcls/ds/ana/sw/conda1-v3/manage/bin/psconda.sh
conda deactivate
conda activate conda_build
(conda_build2) [psreldev@sdfiana002 manage]$ conda --version
conda 23.10.0
source /sdf/group/lcls/ds/ana/sw/conda1-v3/manage/bin/psconda.sh
content changes for conda1-v3
- eval "$(/sdf/group/lcls/ds/ana/sw/conda1/inst/bin/conda shell.bash hook)"
+ eval "$(/sdf/group/lcls/ds/ana/sw/conda1-v3/inst/bin/conda shell.bash hook)"
...
- eval "$(/cds/sw/ds/ana/conda1-v2/inst/bin/conda shell.bash hook)"
+ eval "$(/cds/sw/ds/ana/conda1-v3/inst/bin/conda shell.bash hook)"
~psreldev/git/psana1-feedstock/recipe/meta.yaml
accessible in ~psreldev/git/psana1-feedstock/recipe/meta.yaml (ADD -v3 !!!):
file:///sdf/group/lcls/ds/ana/sw/conda1-v3/manage/downloads/anarel/psana-conda-4.0.56.tar.gz
Conda build
Everything works on s3df psana node under login psreldev because it needs some disk space... Other than that it can be built under any. other user.
Clone psana1-feedstock
IT IS ALREADY DONE ONCE for psreldev
> s3dflogin
ssh psana -l psreldev
cd git
rm -rf psana1-feedstock/
git clone https://github.com/slac-lcls/psana1-feedstock
Installing conda-build in the conda base
source /sdf/group/lcls/ds/ana/sw/conda1/manage/bin/psconda.sh
conda deactivate # > base
conda activate base
conda install conda-build
conda install conda-verify # also works in the base
Set environment
source /sdf/group/lcls/ds/ana/sw/conda1/manage/bin/psconda.sh
conda deactivate
conda activate conda_build
Run conda build
cd ~/git/psana1-feedstock/
Update in ~psreldev/git/psana1-feedstock/recipe/meta.yaml fields for set version and sha256
conda build -c lcls-i -c conda-forge recipe
Deprecated
pdsdata and psalg migrated from svn to github
svn co file:///afs/slac/g/pcds/svn/pdsdata/tags/V09-04-00 extpkgs/pdsdata
svn co file:///afs/slac/g/pcds/svn/psalg/tags/V01-00-12 extpkgs/psalg
created drepositories on git@github.com:lcls-psana/
git clone git@github.com:lcls-psana/pdsdata.git
git clone git@github.com:lcls-psana/psalg.git
create conda_build_svn environment
s3dfnx - nx - server on s3df connects to dubrovin@sdflogin002
. /sdf/group/lcls/ds/ana/sw/conda1/manage/bin/psconda.sh
conda create -n conda_build_svn svn anaconda-client
conda activate and run ana-rel-admin
source /sdf/group/lcls/ds/ana/sw/conda1/manage/bin/psconda.sh
conda deactivate
conda activate conda_build_svn
Remove tag from local git and github
git tag -d V10-00-01 # removes tag on local git
git push --delete origin V10-00-01 # removes tag on github
List of commands to build psana1 release on s3df
Login and set environment
> s3dflogin
ssh psana -l psreldev
source /sdf/group/lcls/ds/ana/sw/conda1-v3/manage/bin/psconda.sh # conda1-v3 is a latest version of conda 23.10.0
conda deactivate
conda activate conda_build
Generate .tar.gz file with source code
cd /sdf/group/lcls/ds/ana/sw/conda1-v3/manage/ # if needed git clone git@github.com:slaclab/anarel-manage.git manage
bin/ana-rel-admin --force --cmd psana-conda-src --name 4.0.58 --basedir `pwd`
Build release
cd ~psreldev/git/psana1-feedstock/ # if needed git clone git@github.com:slac-lcls/psana1-feedstock.git
Update in recipe/meta.yaml fields for set version and sha256
conda build -c lcls-i -c conda-forge recipe
Debugging
In case of problem with tests look at log file like
/sdf/group/lcls/ds/ana/sw/conda1/inst/envs/conda_build2/conda-bld/psana_<13-digit-build number>/test_tmp/work/<log-file-name>
Upload .tar.gz file with release to anaconda lcls-i channel
anaconda upload -u lcls-i /sdf/group/lcls/ds/ana/sw/conda1/inst/envs/conda_build2/conda-bld/linux-64/psana-4.0.57-py39hed0727e_1.tar.bz2
(conda_build2) [psreldev@sdfiana001 psana1-feedstock]$ pwd
/sdf/home/p/psreldev/git/psana1-feedstock
(conda_build2) [psreldev@sdfiana001 psana1-feedstock]$ anaconda upload -u lcls-i /sdf/group/lcls/ds/ana/sw/conda1/inst/envs/conda_build2/conda-bld/linux-64/psana-4.0.57-py39hed0727e_1.tar.bz2
Using Anaconda API: https://api.anaconda.org
Using "lcls-i" as upload username
Processing "/sdf/group/lcls/ds/ana/sw/conda1/inst/envs/conda_build2/conda-bld/linux-64/psana-4.0.57-py39hed0727e_1.tar.bz2"
Detecting file type...
File type is "Conda"
Extracting conda attributes for upload
Creating package "psana"
Creating release "4.0.57"
The action you are performing requires authentication, please sign in:
Using Anaconda API: https://api.anaconda.org
Username: dubrovin
dubrovin's Password:
login successful
Using Anaconda API: https://api.anaconda.org
Using "lcls-i" as upload username
Processing "/sdf/group/lcls/ds/ana/sw/conda1/inst/envs/conda_build2/conda-bld/linux-64/psana-4.0.57-py39hed0727e_1.tar.bz2"
Detecting file type...
File type is "Conda"
Extracting conda attributes for upload
Creating package "psana"
Creating release "4.0.57"
Uploading file "lcls-i/psana/4.0.57/linux-64/psana-4.0.57-py39hed0727e_1.tar.bz2"
15.4MB [00:01, 12.6MB/s]
Upload complete
conda located at:
https://anaconda.org/lcls-i/psana
(conda_build2) [psreldev@sdfiana001 psana1-feedstock]$
Create new environment
conda create --name ana-4.0.58-py3 --clone ana-4.0.57-py3
(base) [psreldev@sdfiana002 psana1-feedstock]$ conda create --name ana-4.0.58-py3 --clone ana-4.0.57-py3
Retrieving notices: ...working... done
Source: /sdf/group/lcls/ds/ana/sw/conda1/inst/envs/ana-4.0.57-py3
Destination: /sdf/group/lcls/ds/ana/sw/conda1/inst/envs/ana-4.0.58-py3
Packages: 488
Files: 11
Downloading and Extracting Packages:
Downloading and Extracting Packages:
Preparing transaction: done
Verifying transaction: done
Executing transaction: /
For Linux 64, Open MPI is built with CUDA awareness but this support is disabled by default.
To enable it, please set the environment variable OMPI_MCA_opal_cuda_support=true before
launching your MPI processes. Equivalently, you can set the MCA parameter in the command line:
mpiexec --mca opal_cuda_support 1 ...
In addition, the UCX support is also built but disabled by default.
To enable it, first install UCX (conda install -c conda-forge ucx). Then, set the environment
variables OMPI_MCA_pml="ucx" OMPI_MCA_osc="ucx" before launching your MPI processes.
Equivalently, you can set the MCA parameters in the command line:
mpiexec --mca pml ucx --mca osc ucx ...
Note that you might also need to set UCX_MEMTYPE_CACHE=n for CUDA awareness via UCX.
Please consult UCX's documentation for detail.
done
#
# To activate this environment, use
#
# $ conda activate ana-4.0.58-py3
#
# To deactivate an active environment, use
#
# $ conda deactivate
(base) [psreldev@sdfiana002 psana1-feedstock]$
conda deactivate
conda activate ana-4.0.58-py3
conda install -c lcls-i -c conda-forge psana=4.0.58
(ana-4.0.58-py3) [psreldev@sdfiana002 psana1-feedstock]$ conda install -c lcls-i -c conda-forge psana=4.0.58
Channels:
- lcls-i
- conda-forge
- defaults
- lcls-ii
- cogsci
Platform: linux-64
Collecting package metadata (repodata.json): done
Solving environment: done
## Package Plan ##
environment location: /sdf/group/lcls/ds/ana/sw/conda1/inst/envs/ana-4.0.58-py3
added / updated specs:
- psana=4.0.58
The following packages will be downloaded:
package | build
---------------------------|-----------------
psana-4.0.58 | py39hed0727e_1 15.4 MB lcls-i
------------------------------------------------------------
Total: 15.4 MB
The following packages will be UPDATED:
psana 4.0.57-py39hed0727e_1 --> 4.0.58-py39hed0727e_1
Proceed ([y]/n)? y
Downloading and Extracting Packages:
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
(ana-4.0.58-py3) [psreldev@sdfiana002 psana1-feedstock]$
Create the same release on pcds
Info from Valerio
activate ana-4.0.58-py3 on S3DF,
>s3dflogin
ssh psana -l psreldev
source /sdf/group/lcls/ds/ana/sw/conda1/manage/bin/psconda.sh
conda deactivate
conda env list
conda activate ana-4.0.58-py3
then:
conda list --explicit > ana-4.0.58-py3.txt
copy ana-4.0.58-py3.txt to psreldev on PCDS - do this trick due to permissions:
>s3dflogin
cp ~psreldev/ana-4.0.61-py3.txt .
> pslogin
ssh psbuild-rhel7-01 -l psreldev
scp dubrovin@s3dflogin.slac.stanford.edu:ana-4.0.61-py3.txt . # complete copy of ana-4.0.61-py3.txt
source /cds/sw/ds/ana/conda1/manage/bin/psconda.sh
conda create -n ana-4.0.58-py3 --file ana-4.0.58-py3.txt
(ana-4.0.54-py3) [psreldev@psbuild-rhel7-01 ~]$ conda create -n ana-4.0.58-py3 --file ana-4.0.58-py3.txt
Downloading and Extracting Packages
plotly-5.13.1 | ############################################################################################################################################################## | 100%
gdk-pixbuf-2.42.10 | ############################################################################################################################################################## | 100%
graphviz-7.1.0 | ############################################################################################################################################################## | 100%
ipykernel-6.20.2 | ############################################################################################################################################################## | 100%
psana-4.0.58 | ############################################################################################################################################################## | 100%
Preparing transaction: done
Verifying transaction: done
Executing transaction: -
For Linux 64, Open MPI is built with CUDA awareness but this support is disabled by default.
To enable it, please set the environment variable OMPI_MCA_opal_cuda_support=true before
launching your MPI processes. Equivalently, you can set the MCA parameter in the command line:
mpiexec --mca opal_cuda_support 1 ...
In addition, the UCX support is also built but disabled by default.
To enable it, first install UCX (conda install -c conda-forge ucx). Then, set the environment
variables OMPI_MCA_pml="ucx" OMPI_MCA_osc="ucx" before launching your MPI processes.
Equivalently, you can set the MCA parameters in the command line:
mpiexec --mca pml ucx --mca osc ucx ...
Note that you might also need to set UCX_MEMTYPE_CACHE=n for CUDA awareness via UCX.
Please consult UCX's documentation for detail.
done
(ana-4.0.54-py3) [psreldev@psbuild-rhel7-01 ~]$
Install new release on pcds:
conda install --experimental-solver=libmamba -c lcls-i -c conda-forge psana=4.0.58 # RECOMMENDED
conda install -c lcls-i -c conda-forge psana=4.0.58 # WORKING
edit and change release name in
pcds: /cds/sw/ds/ana/conda1/manage/bin/psconda.sh
s3df: /sdf/group/lcls/ds/ana/sw/conda1/manage/bin/psconda.sh > conda1-v3
Update release for Jupyter Notebook
Info from Chris:
ssh to pcds node > pslogin
ssh psbuild-rhel7-01 -l psreldev
Update version in TWO lines:
/cds/group/psdm/sw/conda/jhub_config/prod-rhel7/kernels/ana1-current-py3/kernel.json
on pcds: /cds/sw/ds/ana/conda1/inst/envs/<environment>/data/anarelinfo/psana-conda-tags
on s3df: /sdf/group/lcls/ds/ana/sw/conda1/inst/envs/ana-4.0.62-py3/data/anarelinfo/psana-conda-tags
Detector conda_branch=False repo=psdm subdir=Non tag=V02-00-84 ?????
References
