Notes about building psana releases.

psana1

Assemble all packages source code in tar.gz file

pslogin
ssh psbuild-rhel7-01 -l psreldev

Set environment:

source /cds/sw/ds/ana/conda1/inst/etc/profile.d/conda.sh

conda activate conda_build

cd /cds/sw/ds/ana/conda1/manage

look at subdirectory scratch/  to see what is the latest release, e.g. psana-conda-4.0.55, then create next:

bin/ana-rel-admin --force --cmd psana-conda-src --name 4.0.56 --basedir `pwd`     --tagsfile psana-conda-svn-pkgs[-test]

Algorithm of ana-rel-admin

Command bin/ana-rel-admin ... assembles all the source code from github and svn archives with the latest tags into a .tar.gz file

--basedir`pwd` is converted to /cds/sw/ds/ana/conda1/manage/

input: < config/psana-conda-svn-pkgs # contains the list of packages with source code

output:

> scratch/psana-conda-4.0.56/<packages>   # clones of all packages form git or svn

> git clone git@github.com:lcls-psana/data_test.git /tmp/data_test
> scratch/psana-conda-4.0.56/psana-conda-tag    # tags extended with version like 'tag': 'V02-00-76'
> scratch/psana-conda-4.0.56/psana-conda-4.0.56/.sit_release    #  contains psana-conda-4.0.56
> downloads/anarel/psana-conda-4.0.56.tar.gz  # archived content of scratch/psana-conda-4.0.56

> dumps at the end sha256sum .../psana-conda-4.0.56.tar.gz > 443440bf7a7fdbcdf89314b68c07f97f5a89dff38d6da98685e74f0671fac37d

Command bin/ana-rel-admin creates file accessible through url:
file:///cds/sw/ds/ana/conda1/manage/downloads/anarel/psana-conda-4.0.56.tar.gz

sha256sum /cds/sw/ds/ana/conda1/manage/downloads/anarel/psana-conda-4.0.56.tar.gz

> 443440bf7a7fdbcdf89314b68c07f97f5a89dff38d6da98685e74f0671fac37d

Conda build

ssh pslogin (not as psreldev yet...)

ssh psbuild-rhel7-01 -l psreldev

cd git/psana1-feedstock/

Update in  ~psreldev/git/psana1-feedstock/recipe/meta.yaml fields for set version and sha256:

edit psana1-feedstock/recipe/meta.yaml
{% set version = '4.0.56' %}
 ...
 sha256: 443440bf7a7fdbcdf89314b68c07f97f5a89dff38d6da98685e74f0671fac37d
 ...

Set environment:

          source /cds/sw/ds/ana/conda1/inst/etc/profile.d/conda.sh             # OR conda deactivate after previous conda_build

conda activate conda_build_py3

Execute command:

conda build -c lcls-i -c conda-forge recipe

> /cds/home/p/psreldev/conda-bld/linux-64/psana-4.0.56-py39hb869b97_1.tar.bz2

conda build - the tail of output
    xorg-xf86vidmodeproto:     2.3.1-h7f98852_1002          conda-forge
    xorg-xproto:               7.0.31-h7f98852_1007         conda-forge
    xz:                        5.2.6-h166bdaf_0             conda-forge
    zeromq:                    4.3.5-h59595ed_0             conda-forge
    zipp:                      3.17.0-pyhd8ed1ab_0          conda-forge
    zlib:                      1.2.13-hd590300_5            conda-forge
    zstd:                      1.5.5-hfc55251_0             conda-forge

Preparing transaction: ...working... done
Verifying transaction: ...working... done
Executing transaction: ...working... done
export PREFIX=/cds/home/p/psreldev/conda-bld/psana_1700187012881/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl
export SRC_DIR=/cds/home/p/psreldev/conda-bld/psana_1700187012881/test_tmp
+ psana -h

Usage: psana [options] [dataset ...]

  Available options:
    {-h|-?|--help    }         print help message
    {-v|--verbose    } (incr)  verbose output, multiple allowed (initial: 0)
    {-q|--quiet      } (incr)  quieter output, multiple allowed (initial: 2)
    {-b|--calib-dir  } path    calibration directory name, may include {exp} and {instr}, if left empty then do not do calibrations (default: "")
    {-c|--config     } path    configuration file, by default use psana.cfg if it exists (default: "")
    {-e|--experiment } string  experiment name, format: XPP:xpp12311 or xpp12311, by default guess it from data (default: "")
    {-j|--job-name   } string  job name, default is to generate from input file names (default: "")
    {-m|--module     } name    module name, more than one possible
    {-n|--num-events } number  maximum number of events to process, 0 means all (default: 0)
    {-s|--skip-events} number  number of events to skip (default: 0)
    {-p|--num-cpu    } number  number greater than 0 enables multi-processing (default: 0)
    {-o|--option     } string  configuration options, format: module.option[=value]

  Positional parameters:
    dataset - input dataset specification (list of file names or exp=cxi12345:run=123:...)

+ exit 0

Resource usage statistics from testing psana:
   Process count: 1
   CPU time: Sys=0:00:00.1, User=-
   Memory: 1.4M
   Disk usage: 12B
   Time elapsed: 0:00:02.4

TEST END: /cds/home/p/psreldev/conda-bld/linux-64/psana-4.0.56-py39hb869b97_1.tar.bz2
Renaming work directory '/cds/home/p/psreldev/conda-bld/psana_1700187012881/work' to '/cds/home/p/psreldev/conda-bld/psana_1700187012881/work_moved_psana-4.0.56-py39hb869b97_1_linux-64_main_build_loop'
INFO:conda_build.utils:Renaming work directory '/cds/home/p/psreldev/conda-bld/psana_1700187012881/work' to '/cds/home/p/psreldev/conda-bld/psana_1700187012881/work_moved_psana-4.0.56-py39hb869b97_1_linux-64_main_build_loop'
INFO conda_build.utils:shutil_move_more_retrying(2075): Renaming work directory '/cds/home/p/psreldev/conda-bld/psana_1700187012881/work' to '/cds/home/p/psreldev/conda-bld/psana_1700187012881/work_moved_psana-4.0.56-py39hb869b97_1_linux-64_main_build_loop'
shutil.move(work)=/cds/home/p/psreldev/conda-bld/psana_1700187012881/work, dest=/cds/home/p/psreldev/conda-bld/psana_1700187012881/work_moved_psana-4.0.56-py39hb869b97_1_linux-64_main_build_loop)
INFO:conda_build.utils:shutil.move(work)=/cds/home/p/psreldev/conda-bld/psana_1700187012881/work, dest=/cds/home/p/psreldev/conda-bld/psana_1700187012881/work_moved_psana-4.0.56-py39hb869b97_1_linux-64_main_build_loop)
INFO conda_build.utils:shutil_move_more_retrying(2082): shutil.move(work)=/cds/home/p/psreldev/conda-bld/psana_1700187012881/work, dest=/cds/home/p/psreldev/conda-bld/psana_1700187012881/work_moved_psana-4.0.56-py39hb869b97_1_linux-64_main_build_loop)
# Automatic uploading is disabled
# If you want to upload package(s) to anaconda.org later, type:

anaconda upload /cds/home/p/psreldev/conda-bld/linux-64/psana-4.0.56-py39hb869b97_1.tar.bz2

# To have conda build upload to anaconda.org automatically, use
# $ conda config --set anaconda_upload yes
anaconda_upload is not set.  Not uploading wheels: []
####################################################################################
Resource usage summary:

Total time: 1:22:33.6
CPU usage: sys=0:03:04.5, user=0:27:14.5
Maximum memory usage observed: 1.6G
Total disk usage observed (not including envs): 285.6M

####################################################################################
Source and build intermediates have been left in /cds/home/p/psreldev/conda-bld.
There are currently 1 accumulated.
To remove them, you can run the ```conda build purge``` command
(conda_build) [psreldev@psbuild-rhel7-01 psana1-feedstock]$

Upload the file to anaconda lcls-i channel

anaconda upload -u lcls-i /sdf/group/lcls/ds/ana/sw/conda1/inst/envs/conda_build2/conda-bld/linux-64/psana-4.0.57-py39hed0727e_1.tar.bz2

anaconda upload messages
(conda_build2) [psreldev@sdfiana001 psana1-feedstock]$ pwd
/sdf/home/p/psreldev/git/psana1-feedstock
(conda_build2) [psreldev@sdfiana001 psana1-feedstock]$ anaconda upload -u lcls-i /sdf/group/lcls/ds/ana/sw/conda1/inst/envs/conda_build2/conda-bld/linux-64/psana-4.0.57-py39hed0727e_1.tar.bz2
Using Anaconda API: https://api.anaconda.org
Using "lcls-i" as upload username
Processing "/sdf/group/lcls/ds/ana/sw/conda1/inst/envs/conda_build2/conda-bld/linux-64/psana-4.0.57-py39hed0727e_1.tar.bz2"
Detecting file type...
File type is "Conda"
Extracting conda attributes for upload
Creating package "psana"
Creating release "4.0.57"
The action you are performing requires authentication, please sign in:
Using Anaconda API: https://api.anaconda.org
Username: dubrovin
dubrovin's Password: 
login successful
Using Anaconda API: https://api.anaconda.org
Using "lcls-i" as upload username
Processing "/sdf/group/lcls/ds/ana/sw/conda1/inst/envs/conda_build2/conda-bld/linux-64/psana-4.0.57-py39hed0727e_1.tar.bz2"
Detecting file type...
File type is "Conda"
Extracting conda attributes for upload
Creating package "psana"
Creating release "4.0.57"
Uploading file "lcls-i/psana/4.0.57/linux-64/psana-4.0.57-py39hed0727e_1.tar.bz2"
15.4MB [00:01, 12.6MB/s]                                                                                                                                                                                  
Upload complete

conda located at:
  https://anaconda.org/lcls-i/psana

(conda_build2) [psreldev@sdfiana001 psana1-feedstock]$

Create new environment

goal: create a real 4.0.57

conda create --name psana-conda-4.0.57 --clone psana-conda-4.0.56

log of conda create
(conda_build2) [psreldev@sdfiana001 psana1-feedstock]$ conda create --name ana-4.0.57-py3 --clone ana-4.0.55-py3
Source:      /sdf/group/lcls/ds/ana/sw/conda1/inst/envs/ana-4.0.55-py3
Destination: /sdf/group/lcls/ds/ana/sw/conda1/inst/envs/ana-4.0.57-py3
Packages: 488
Files: 11

Downloading and Extracting Packages:
font-ttf-dejavu-sans | ########################################################################################################################################################################### | 100%^                                                                                                                                                                                                          
                                                                                                                                                                                                          
Downloading and Extracting Packages:                                                                                                                                                                      
                                                                                                                                                                                                          
Preparing transaction: done                                                                                                                                                                               
Verifying transaction: done                                                                                                                                                                               
Executing transaction: |                                                                                                                                                                                  
For Linux 64, Open MPI is built with CUDA awareness but this support is disabled by default.                                                                                                              
To enable it, please set the environment variable OMPI_MCA_opal_cuda_support=true before                                                                                                                  
launching your MPI processes. Equivalently, you can set the MCA parameter in the command line:                                                                                                            
mpiexec --mca opal_cuda_support 1 ...                                                                                                                                                                     
                                                                                                                                                                                                          
In addition, the UCX support is also built but disabled by default.                                                                                                                                       
To enable it, first install UCX (conda install -c conda-forge ucx). Then, set the environment                                                                                                             
variables OMPI_MCA_pml="ucx" OMPI_MCA_osc="ucx" before launching your MPI processes.                                                                                                                      
Equivalently, you can set the MCA parameters in the command line:                                                                                                                                         
mpiexec --mca pml ucx --mca osc ucx ...                                                                                                                                                                   
Note that you might also need to set UCX_MEMTYPE_CACHE=n for CUDA awareness via UCX.                                                                                                                      
Please consult UCX's documentation for detail.                                                                                                                                                            
                                                                                                                                                                                                          
                                                                                                                                                                                                          
done                                                                                                                                                                                                      
#                                                                                                                                                                                                         
# To activate this environment, use                                                                                                                                                                       
#                                                                                                                                                                                                         
#     $ conda activate ana-4.0.57-py3                                                                                                                                                                     
#                                                                                                                                                                                                         
# To deactivate an active environment, use                                                                                                                                                                
#                                                                                                                                                                                                         
#     $ conda deactivate                                                                                                                                                                                  
                                                                                                                                                                                                          
(conda_build2) [psreldev@sdfiana001 psana1-feedstock]$ 


conda deactivate
conda activate psana-conda-4.0.57
conda install -c lcls-i -c conda-forge psana=4.0.57  # Chris? --experimental-solver=libmamba

have to do this on both old-psana and new-s3df as psreldev

Extra operetions

See details in ~/.bash_history or ~/.k5login

cd /cds/sw/ds/ana/conda1/

if ./manage is not available

    git clone git@github.com:slaclab/anarel-manage.git manage

cd manage

update it if necessary

git pull --rebase

conda create -n conda_build_py3 python=3.9 anaconda

conda env remove --name conda_build_py3

see /cds/sw/ds/ana/conda1/inst/envs/ OR conda info --envs

ssh psbuild-rhel7-01 -l psreldev

cd git/psana1-feedstock/

sha256sum /cds/sw/ds/ana/conda1/manage/downloads/anarel/psana-conda-4.0.56.tar.gz

> 443440bf7a7fdbcdf89314b68c07f97f5a89dff38d6da98685e74f0671fac37d

~psreldev/git/psana1-feedstock/recipe/meta.yaml

cd git
rm -rf psana1-feedstock/
git clone https://github.com/slac-lcls/psana1-feedstock

Set environment (chris):

source /cds/sw/ds/ana/conda2/manage/bin/psconda.sh.  # Chris: conda2 because it needs py3, but conda1 also works with py3...
conda deactivate
conda activate conda_build

OR:

source /cds/sw/ds/ana/conda2/manage/bin/psconda.sh

conda create -n conda_build_py3 python conda-build anaconda-client

conda activate conda_build_py3

ssh psbuild-rhel7-01 -l psreldev

cd git
rm -rf psana1-feedstock/
git clone https://github.com/slac-lcls/psana1-feedstock

cp /cds/sw/ds/ana/conda1/manage/downloads/anarel/psana-conda-4.0.56.tar.gz /reg/g/psdm/web/swdoc/tutorials/


Issue: PermissionError: [Errno 13] Permission denied: '/cds/sw/ds/ana/conda2/inst/envs/conda_build/pkgs/libselinux-cos6-x86_64-2.0.94-h9b0a68f_1105'

Chris: cp ~cpo/.condarc ~/. # if ~/.condarc is not available

Valerio: conda create -n local_conda_build python conda-build conda-validate anaconda-client

scons test-Detector

  • git repository data_test_access cloned with all other psana packages
  • location for data_test can be controlled in config/psana-conda-svn-pkgs:
    • subdir=/tmp - clones this package as /tmp/data_test
  • data_test_access is included but data_test is ignored in tar cfvz /cds/sw/ds/ana/conda1/manage/downloads/anarel/psana-conda-4.0.56.tar.gz psana-conda-4.0.56

Build on s3df

Initialization for build on s3df

IT IS ALREADY DONE ONCE

clone anarel-manage

cd .../con-build     OR   cd /sdf/group/lcls/ds/ana/sw/conda1/
. /sdf/group/lcls/ds/ana/sw/conda1/manage/bin/psconda.sh

git clone git@github.com:slaclab/anarel-manage.git

make github seen from psana nodes

ema ~/.gitconfig

\[http\] - remove

Run bin/ana-rel-admin on s3df

login as myself

> s3dfnx ends up as dubrovin@sdflogin002

> psana ... because /afs/slac/g/pcds/svn/pdsdata is seen from psana nodes only... (Wilko)

cd <path-to>/anarel-manage

login as psreldev

> s3dfnx ends up as dubrovin@sdflogin002

ssh psana -l psreldev

cd /sdf/group/lcls/ds/ana/sw/conda1/manage

conda activate and run ana-rel-admin 

source /sdf/group/lcls/ds/ana/sw/conda1/manage/bin/psconda.sh

conda deactivate

conda activate conda_build

bin/ana-rel-admin --force --cmd psana-conda-src --name 4.0.56 --basedir `pwd`

Command bin/ana-rel-admin creates file

/sdf/group/lcls/ds/ana/sw/conda1/manage/downloads/anarel/psana-conda-4.0.56.tar.gz

accessible in ~psreldev/git/psana1-feedstock/recipe/meta.yaml through url:

file:///sdf/group/lcls/ds/ana/sw/conda1/manage/downloads/anarel/psana-conda-4.0.56.tar.gz

conda1-v3

Valerio generated new conda1-v3

source /sdf/group/lcls/ds/ana/sw/conda1-v3/manage/bin/psconda.sh
conda deactivate
conda activate conda_build

(conda_build2) [psreldev@sdfiana002 manage]$ conda --version
conda 23.10.0


source /sdf/group/lcls/ds/ana/sw/conda1-v3/manage/bin/psconda.sh

content changes for conda1-v3

-    eval "$(/sdf/group/lcls/ds/ana/sw/conda1/inst/bin/conda shell.bash hook)"
+    eval "$(/sdf/group/lcls/ds/ana/sw/conda1-v3/inst/bin/conda shell.bash hook)"

...

-    eval "$(/cds/sw/ds/ana/conda1-v2/inst/bin/conda shell.bash hook)"
+    eval "$(/cds/sw/ds/ana/conda1-v3/inst/bin/conda shell.bash hook)"


~psreldev/git/psana1-feedstock/recipe/meta.yaml

accessible in ~psreldev/git/psana1-feedstock/recipe/meta.yaml (ADD -v3 !!!):

file:///sdf/group/lcls/ds/ana/sw/conda1-v3/manage/downloads/anarel/psana-conda-4.0.56.tar.gz


Conda build

Everything works on s3df psana node under login psreldev because it needs some disk space... Other than that it can be built under any. other user.

Clone psana1-feedstock

IT IS ALREADY DONE ONCE for psreldev

> s3dflogin

ssh psana -l psreldev

cd git
rm -rf psana1-feedstock/
git clone https://github.com/slac-lcls/psana1-feedstock

Installing conda-build in the conda base

source /sdf/group/lcls/ds/ana/sw/conda1/manage/bin/psconda.sh

conda deactivate # > base

conda activate base

conda install conda-build
conda install conda-verify # also works in the base

Set environment

source /sdf/group/lcls/ds/ana/sw/conda1/manage/bin/psconda.sh

conda deactivate

conda activate conda_build

Run conda build

cd ~/git/psana1-feedstock/

Update in  ~psreldev/git/psana1-feedstock/recipe/meta.yaml fields for set version and sha256

conda build -c lcls-i -c conda-forge recipe

Deprecated

pdsdata and psalg migrated from svn to github

svn co file:///afs/slac/g/pcds/svn/pdsdata/tags/V09-04-00 extpkgs/pdsdata
svn co file:///afs/slac/g/pcds/svn/psalg/tags/V01-00-12 extpkgs/psalg

created drepositories on git@github.com:lcls-psana/

git clone git@github.com:lcls-psana/pdsdata.git
git clone git@github.com:lcls-psana/psalg.git

create conda_build_svn environment

s3dfnx - nx - server on s3df connects to dubrovin@sdflogin002

. /sdf/group/lcls/ds/ana/sw/conda1/manage/bin/psconda.sh

conda create -n conda_build_svn svn anaconda-client

conda activate and run ana-rel-admin 

source /sdf/group/lcls/ds/ana/sw/conda1/manage/bin/psconda.sh

conda deactivate

conda activate conda_build_svn

Remove tag from local git and github

git tag -d V10-00-01  # removes tag on local git

git push --delete origin V10-00-01  # removes tag on github

List of commands to build psana1 release on s3df

Login and set environment

> s3dflogin

ssh psana -l psreldev

source /sdf/group/lcls/ds/ana/sw/conda1-v3/manage/bin/psconda.sh  # conda1-v3 is a latest version of conda 23.10.0
conda deactivate
conda activate conda_build

Generate .tar.gz file with source code

cd /sdf/group/lcls/ds/ana/sw/conda1-v3/manage/  # if needed git clone git@github.com:slaclab/anarel-manage.git manage

bin/ana-rel-admin --force --cmd psana-conda-src --name 4.0.58 --basedir `pwd`

Build release

cd ~psreldev/git/psana1-feedstock/   # if needed git clone git@github.com:slac-lcls/psana1-feedstock.git

Update in  recipe/meta.yaml fields for set version and sha256

conda build -c lcls-i -c conda-forge recipe

Debugging

In case of problem with tests look at log file like

/sdf/group/lcls/ds/ana/sw/conda1/inst/envs/conda_build2/conda-bld/psana_<13-digit-build number>/test_tmp/work/<log-file-name>

Upload .tar.gz file with release to anaconda lcls-i channel

anaconda upload -u lcls-i /sdf/group/lcls/ds/ana/sw/conda1/inst/envs/conda_build2/conda-bld/linux-64/psana-4.0.57-py39hed0727e_1.tar.bz2

response on anaconda upload
(conda_build2) [psreldev@sdfiana001 psana1-feedstock]$ pwd
/sdf/home/p/psreldev/git/psana1-feedstock
(conda_build2) [psreldev@sdfiana001 psana1-feedstock]$ anaconda upload -u lcls-i /sdf/group/lcls/ds/ana/sw/conda1/inst/envs/conda_build2/conda-bld/linux-64/psana-4.0.57-py39hed0727e_1.tar.bz2
Using Anaconda API: https://api.anaconda.org
Using "lcls-i" as upload username
Processing "/sdf/group/lcls/ds/ana/sw/conda1/inst/envs/conda_build2/conda-bld/linux-64/psana-4.0.57-py39hed0727e_1.tar.bz2"
Detecting file type...
File type is "Conda"
Extracting conda attributes for upload
Creating package "psana"
Creating release "4.0.57"
The action you are performing requires authentication, please sign in:
Using Anaconda API: https://api.anaconda.org
Username: dubrovin
dubrovin's Password: 
login successful
Using Anaconda API: https://api.anaconda.org
Using "lcls-i" as upload username
Processing "/sdf/group/lcls/ds/ana/sw/conda1/inst/envs/conda_build2/conda-bld/linux-64/psana-4.0.57-py39hed0727e_1.tar.bz2"
Detecting file type...
File type is "Conda"
Extracting conda attributes for upload
Creating package "psana"
Creating release "4.0.57"
Uploading file "lcls-i/psana/4.0.57/linux-64/psana-4.0.57-py39hed0727e_1.tar.bz2"
15.4MB [00:01, 12.6MB/s]                                                                                                                                                                                  
Upload complete

conda located at:
  https://anaconda.org/lcls-i/psana

(conda_build2) [psreldev@sdfiana001 psana1-feedstock]$

Create new environment

conda create --name ana-4.0.58-py3 --clone ana-4.0.57-py3

response on conda create
(base) [psreldev@sdfiana002 psana1-feedstock]$ conda create --name ana-4.0.58-py3 --clone ana-4.0.57-py3
Retrieving notices: ...working... done
Source:      /sdf/group/lcls/ds/ana/sw/conda1/inst/envs/ana-4.0.57-py3
Destination: /sdf/group/lcls/ds/ana/sw/conda1/inst/envs/ana-4.0.58-py3
Packages: 488
Files: 11

Downloading and Extracting Packages:

Downloading and Extracting Packages:

Preparing transaction: done
Verifying transaction: done
Executing transaction: /  
For Linux 64, Open MPI is built with CUDA awareness but this support is disabled by default.
To enable it, please set the environment variable OMPI_MCA_opal_cuda_support=true before
launching your MPI processes. Equivalently, you can set the MCA parameter in the command line:
mpiexec --mca opal_cuda_support 1 ...
 
In addition, the UCX support is also built but disabled by default.
To enable it, first install UCX (conda install -c conda-forge ucx). Then, set the environment
variables OMPI_MCA_pml="ucx" OMPI_MCA_osc="ucx" before launching your MPI processes.
Equivalently, you can set the MCA parameters in the command line:
mpiexec --mca pml ucx --mca osc ucx ...
Note that you might also need to set UCX_MEMTYPE_CACHE=n for CUDA awareness via UCX.
Please consult UCX's documentation for detail.
 
done
#
# To activate this environment, use
#
#     $ conda activate ana-4.0.58-py3
#
# To deactivate an active environment, use
#
#     $ conda deactivate

(base) [psreldev@sdfiana002 psana1-feedstock]$

conda deactivate
conda activate ana-4.0.58-py3
conda install -c lcls-i -c conda-forge psana=4.0.58

response on conda install
(ana-4.0.58-py3) [psreldev@sdfiana002 psana1-feedstock]$ conda install -c lcls-i -c conda-forge psana=4.0.58
Channels:
 - lcls-i
 - conda-forge
 - defaults
 - lcls-ii
 - cogsci
Platform: linux-64
Collecting package metadata (repodata.json): done
Solving environment: done

## Package Plan ##

  environment location: /sdf/group/lcls/ds/ana/sw/conda1/inst/envs/ana-4.0.58-py3

  added / updated specs:
    - psana=4.0.58


The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    psana-4.0.58               |   py39hed0727e_1        15.4 MB  lcls-i
    ------------------------------------------------------------
                                           Total:        15.4 MB

The following packages will be UPDATED:

  psana                               4.0.57-py39hed0727e_1 --> 4.0.58-py39hed0727e_1 


Proceed ([y]/n)? y


Downloading and Extracting Packages:
                                                                                                                                                                                                    
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
(ana-4.0.58-py3) [psreldev@sdfiana002 psana1-feedstock]$


Create the same release on pcds


Info from Valerio

activate ana-4.0.58-py3 on S3DF,

>s3dflogin

ssh psana -l psreldev

source /sdf/group/lcls/ds/ana/sw/conda1/manage/bin/psconda.sh

conda deactivate

conda env list

conda activate ana-4.0.58-py3

then:

conda list --explicit > ana-4.0.58-py3.txt

copy ana-4.0.58-py3.txt to psreldev on PCDS - do this trick due to permissions:

>s3dflogin

cp ~psreldev/ana-4.0.61-py3.txt .

> pslogin

ssh psbuild-rhel7-01 -l psreldev

scp dubrovin@s3dflogin.slac.stanford.edu:ana-4.0.61-py3.txt .  # complete copy of ana-4.0.61-py3.txt

source /cds/sw/ds/ana/conda1/manage/bin/psconda.sh

conda create -n ana-4.0.58-py3 --file ana-4.0.58-py3.txt

response on conda create
(ana-4.0.54-py3) [psreldev@psbuild-rhel7-01 ~]$ conda create -n ana-4.0.58-py3 --file ana-4.0.58-py3.txt

Downloading and Extracting Packages
plotly-5.13.1        | ############################################################################################################################################################## | 100% 
gdk-pixbuf-2.42.10   | ############################################################################################################################################################## | 100% 
graphviz-7.1.0       | ############################################################################################################################################################## | 100% 
ipykernel-6.20.2     | ############################################################################################################################################################## | 100% 
psana-4.0.58         | ############################################################################################################################################################## | 100% 
Preparing transaction: done
Verifying transaction: done
Executing transaction: -  
For Linux 64, Open MPI is built with CUDA awareness but this support is disabled by default.
To enable it, please set the environment variable OMPI_MCA_opal_cuda_support=true before
launching your MPI processes. Equivalently, you can set the MCA parameter in the command line:
mpiexec --mca opal_cuda_support 1 ...
 
In addition, the UCX support is also built but disabled by default.
To enable it, first install UCX (conda install -c conda-forge ucx). Then, set the environment
variables OMPI_MCA_pml="ucx" OMPI_MCA_osc="ucx" before launching your MPI processes.
Equivalently, you can set the MCA parameters in the command line:
mpiexec --mca pml ucx --mca osc ucx ...
Note that you might also need to set UCX_MEMTYPE_CACHE=n for CUDA awareness via UCX.
Please consult UCX's documentation for detail.
 

done
(ana-4.0.54-py3) [psreldev@psbuild-rhel7-01 ~]$


Install new release on pcds:

conda install --experimental-solver=libmamba -c lcls-i -c conda-forge psana=4.0.58 # RECOMMENDED

conda install -c lcls-i -c conda-forge psana=4.0.58 # WORKING



edit and change release name in 

pcds: /cds/sw/ds/ana/conda1/manage/bin/psconda.sh

s3df: /sdf/group/lcls/ds/ana/sw/conda1/manage/bin/psconda.sh    > conda1-v3

Update release for Jupyter Notebook

Info from Chris:

ssh to pcds node > pslogin

ssh psbuild-rhel7-01 -l psreldev

Update version in TWO lines:

/cds/group/psdm/sw/conda/jhub_config/prod-rhel7/kernels/ana1-current-py3/kernel.json


Tags of packages included in the release

on pcds: /cds/sw/ds/ana/conda1/inst/envs/<environment>/data/anarelinfo/psana-conda-tags

on s3df: /sdf/group/lcls/ds/ana/sw/conda1/inst/envs/ana-4.0.62-py3/data/anarelinfo/psana-conda-tags

Detector   conda_branch=False   repo=psdm   subdir=Non    tag=V02-00-84  ?????


References



  • No labels