anaconda upload -u lcls-i /cds/home/p/psreldevsdf/group/lcls/ds/ana/sw/conda1/inst/envs/conda_build2/conda-bld/linux-64/psana-4.0.5657-py39hb869b97py39hed0727e_1.tar.bz2

Code Block

title	anaconda upload messages
collapse	true

(conda_buildbuild2) [psreldev@psbuild-rhel7-01psreldev@sdfiana001 psana1-feedstock]$ pwd
/sdf/home/p/psreldev/git/psana1-feedstock
(conda_build2) [psreldev@sdfiana001 psana1-feedstock]$ anaconda upload /cds/home/p/psreldev-u lcls-i /sdf/group/lcls/ds/ana/sw/conda1/inst/envs/conda_build2/conda-bld/linux-64/psana-4.0.5657-py39hb869b97py39hed0727e_1.tar.bz2
Using Anaconda API: https://api.anaconda.org
Using "chrisvamlcls-i" as upload username
Processing '/cds/home/p/psreldev"/sdf/group/lcls/ds/ana/sw/conda1/inst/envs/conda_build2/conda-bld/linux-64/psana-4.0.5657-py39hb869b97py39hed0727e_1.tar.bz2'"
Detecting file type...
File type is "condaConda"
Extracting conda package attributes for upload
Creating package "psana"
Creating release "4.0.5657"
Uploading file "chrisvam/psana/4.0.56/linux-64/psana-4.0.56-py39hb869b97_1.tar.bz2"
 uploaded 18698 of 18698Kb: 100.00% ETA: 0.0 minutes
Upload complete

conda package located at:
https://anaconda.org/chrisvam/psana

(conda_build) [psreldev@psbuild-rhel7-01 psana1-feedstock]$

Create new environment

goal: create a real 4.0.57

...

The action you are performing requires authentication, please sign in:
Using Anaconda API: https://api.anaconda.org
Username: dubrovin
dubrovin's Password: 
login successful
Using Anaconda API: https://api.anaconda.org
Using "lcls-i" as upload username
Processing "/sdf/group/lcls/ds/ana/sw/conda1/inst/envs/conda_build2/conda-bld/linux-64/psana-4.0.57-py39hed0727e_1.tar.bz2"
Detecting file type...
File type is "Conda"
Extracting conda attributes for upload
Creating package "psana"
Creating release "4.0.57"
Uploading file "lcls-i/psana/4.0.57/linux-64/psana-4.0.57-py39hed0727e_1.tar.bz2"
15.4MB [00:01, 12.6MB/s]                                                                                                                                                                                  
Upload complete

conda located at:
  https://anaconda.org/lcls-i/psana

(conda_build2) [psreldev@sdfiana001 psana1-feedstock]$

Create new environment

goal: create a real 4.0.57

conda create --name psana-conda-4.0.57 --clone psana-conda-4.0.56

Code Block

title	log of conda create
collapse	true

(conda_build2) [psreldev@sdfiana001 psana1-feedstock]$ conda create --name ana-4.0.57-py3 --clone ana-4.0.55-py3
Source:      /sdf/group/lcls/ds/ana/sw/conda1/inst/envs/ana-4.0.55-py3
Destination: /sdf/group/lcls/ds/ana/sw/conda1/inst/envs/ana-4.0.57-py3
Packages: 488
Files: 11

Downloading and Extracting Packages:
font-ttf-dejavu-sans | ########################################################################################################################################################################### | 100%^                                                                                                                                                                                                          
                                                                                                                                                                                                          
Downloading and Extracting Packages:                                                                                                                                                                      
                                                                                                                                                                                                          
Preparing transaction: done                                                                                                                                                                               
Verifying transaction: done                                                                                                                                                                               
Executing transaction: |                                                                                                                                                                                  
For Linux 64, Open MPI is built with CUDA awareness but this support is disabled by default.                                                                                                              
To enable it, please set the environment variable OMPI_MCA_opal_cuda_support=true before                                                                                                                  
launching your MPI processes. Equivalently, you can set the MCA parameter in the command line:                                                                                                            
mpiexec --mca opal_cuda_support 1 ...                                                                                                                                                                     
                                                                                                                                                                                                          
In addition, the UCX support is also built but disabled by default.                                                                                                                                       
To enable it, first install UCX (conda install -c conda-forge ucx). Then, set the environment                                                                                                             
variables OMPI_MCA_pml="ucx" OMPI_MCA_osc="ucx" before launching your MPI processes.                                                                                                                      
Equivalently, you can set the MCA parameters in the command line:                                                                                                                                         
mpiexec --mca pml ucx --mca osc ucx ...                                                                                                                                                                   
Note that you might also need to set UCX_MEMTYPE_CACHE=n for CUDA awareness via UCX.                                                                                                                      
Please consult UCX's documentation for detail.                                                                                                                                                            
                                                                                                                                                                                                          
                                                                                                                                                                                                          
done                                                                                                                                                                                                      
#                                                                                                                                                                                                         
# To activate this environment, use                                                                                                                                                                       
#                                                                                                                                                                                                         
#     $ conda activate ana-4.0.57-py3                                                                                                                                                                     
#                                                                                                                                                                                                         
# To deactivate an active environment, use                                                                                                                                                                
#                                                                                                                                                                                                         
#     $ conda deactivate                                                                                                                                                                                  
                                                                                                                                                                                                          
(conda_build2) [psreldev@sdfiana001 psana1-feedstock]$

conda deactivate
conda activate psana-conda-4.0.57
conda install -c lcls-i -c conda-forge psana=4.0.57 # Chris? --experimental-solver=libmamba

have to do this on both old-psana and new-s3df as psreldev

Extra operetions

See details in ~/.bash_history or ~/.k5login

cd /cds/sw/ds/ana/conda1/

if ./manage is not available

git clone git@github.com:slaclab/anarel-manage.git manage

cd manage

update it if necessary

git pull --rebase

conda create -n conda_build_py3 python=3.9 anaconda

conda env remove --name conda_build_py3

see /cds/sw/ds/ana/conda1/inst/envs/ OR conda info --envs

ssh psbuild-rhel7-01 -l psreldev

cd git/psana1-feedstock/

sha256sum /cds/sw/ds/ana/conda1/manage/downloads/anarel/psana-conda-4.0.56.tar.gz

> 443440bf7a7fdbcdf89314b68c07f97f5a89dff38d6da98685e74f0671fac37d

~psreldev/git/psana1-feedstock/recipe/meta.yaml

cd git
rm -rf psana1-feedstock/
git clone https://github.com/slac-lcls/psana1-feedstock

Set environment (chris):

source /cds/sw/ds/ana/conda2/manage/bin/psconda.sh. # Chris: conda2 because it needs py3, but conda1 also works with py3...
conda deactivate
conda activate conda_build

OR:

source /cds/sw/ds/ana/conda2/manage/bin/psconda.sh

conda create -n conda_build_py3 python conda-build anaconda-client

conda activate conda_build_py3

ssh psbuild-rhel7-01 -l psreldev

cd git
rm -rf psana1-feedstock/
git clone https://github.com/slac-lcls/psana1-feedstock

cp /cds/sw/ds/ana/conda1/manage/downloads/anarel/psana-conda-4.0.56.tar.gz /reg/g/psdm/web/swdoc/tutorials/

copy here because psreldev does not have permission !!!!???
because later it is used in psana1-feedstock/recipe/meta.yaml as url: https://pswww.slac.stanford.edu/swdoc/tutorials/{{ name }}-conda-{{ version }}.tar.gz

Issue: PermissionError: [Errno 13] Permission denied: '/cds/sw/ds/ana/conda2/inst/envs/conda_build/pkgs/libselinux-cos6-x86_64-2.0.94-h9b0a68f_1105'

Chris: cp ~cpo/.condarc ~/. # if ~/.condarc is not available

Valerio: conda create -n local_conda_build python conda-build conda-validate anaconda-client

scons test-Detector

git repository data_test_access cloned with all other psana packages
location for data_test can be controlled in config/psana-conda-svn-pkgs:
- subdir=/tmp - clones this package as /tmp/data_test
data_test_access is included but data_test is ignored in tar cfvz /cds/sw/ds/ana/conda1/manage/downloads/anarel/psana-conda-4.0.56.tar.gz psana-conda-4.0.56

Build on s3df

Initialization for build on s3df

IT IS ALREADY DONE ONCE

clone anarel-manage

cd .../con-build OR cd /sdf/group/lcls/ds/ana/sw/conda1/
. /sdf/group/lcls/ds/ana/sw/conda1/manage/bin/psconda.sh

git clone git@github.com:slaclab/anarel-manage.git

make github seen from psana nodes

ema ~/.gitconfig

\[http\] - remove

Run bin/ana-rel-admin on s3df

login as myself

> s3dfnx ends up as dubrovin@sdflogin002

> psana ... because /afs/slac/g/pcds/svn/pdsdata is seen from psana nodes only... (Wilko)

cd <path-to>/anarel-manage

login as psreldev

> s3dfnx ends up as dubrovin@sdflogin002

ssh psana -l psreldev

cd /sdf/group/lcls/ds/ana/sw/conda1/manage

conda activate and run ana-rel-admin

source /sdf/group/lcls/ds/ana/sw/conda1/manage/bin/psconda.sh

conda deactivate

conda activate conda_build

bin/ana-rel-admin --force --cmd psana-conda-src --name 4.0.56 --basedir `pwd`

Command bin/ana-rel-admin creates file

/sdf/group/lcls/ds/ana/sw/conda1/manage/downloads/anarel/psana-conda-4.0.56.tar.gz

accessible in ~psreldev/git/psana1-feedstock/recipe/meta.yaml through url:

file:///sdf/group/lcls/ds/ana/sw/conda1/manage/downloads/anarel/psana-conda-4.0.56.tar.gz

conda1-v3

Valerio generated new conda1-v3

source /sdf/group/lcls/ds/ana/sw/conda1-v3/manage/bin/psconda.sh
conda deactivate
conda activate conda_build

(conda_build2) [psreldev@sdfiana002 manage]$ conda --version
conda 23.10.0

source /sdf/group/lcls/ds/ana/sw/conda1-v3/manage/bin/psconda.sh

content changes for conda1-v3

- eval "$(/sdf/group/lcls/ds/ana/sw/conda1/inst/bin/conda shell.bash hook)"
+ eval "$(/sdf/group/lcls/ds/ana/sw/conda1-v3/inst/bin/conda shell.bash hook)"

...

- eval "$(/cds/sw/ds/ana/conda1-v2/inst/bin/conda shell.bash hook)"
+ eval "$(/cds/sw/ds/ana/conda1-v3/inst/bin/conda shell.bash hook)"

~psreldev/git/psana1-feedstock/recipe/meta.yaml

accessible in ~psreldev/git/psana1-feedstock/recipe/meta.yaml (ADD -v3 !!!):

file:///sdf/group/lcls/ds/ana/sw/conda1-v3/manage/downloads/anarel/psana-conda-4.0.56.tar.gz

Conda build

Everything works on s3df psana node under login psreldev because it needs some disk space... Other than that it can be built under any. other user.

Clone psana1-feedstock

IT IS ALREADY DONE ONCE for psreldev

> s3dflogin

ssh psana -l psreldev

cd git
rm -rf psana1-feedstock/
git clone https://github.com/slac-lcls/psana1-feedstock

Installing conda-build in the conda base

source /sdf/group/lcls/ds/ana/sw/conda1/manage/bin/psconda.sh

conda deactivate # > base

conda activate base

conda install conda-build
conda install conda-verify # also works in the base

Set environment

source /sdf/group/lcls/ds/ana/sw/conda1/manage/bin/psconda.sh

conda deactivate

conda activate conda_build

Run conda build

cd ~/git/psana1-feedstock/

Update in ~psreldev/git/psana1-feedstock/recipe/meta.yaml fields for set version and sha256

conda build -c lcls-i -c conda-forge recipe

Deprecated

pdsdata and psalg migrated from svn to github

svn co file:///afs/slac/g/pcds/svn/pdsdata/tags/V09-04-00 extpkgs/pdsdata
svn co file:///afs/slac/g/pcds/svn/psalg/tags/V01-00-12 extpkgs/psalg

created drepositories on git@github.com:lcls-psana/

git clone git@github.com:lcls-psana/pdsdata.git
git clone git@github.com:lcls-psana/psalg.git

create conda_build_svn environment

s3dfnx - nx - server on s3df connects to dubrovin@sdflogin002

. /sdf/group/lcls/ds/ana/sw/conda1/manage/bin/psconda.sh

conda create -n conda_build_svn svn anaconda-client

conda activate and run ana-rel-admin

source /sdf/group/lcls/ds/ana/sw/conda1/manage/bin/psconda.sh

conda deactivate

conda activate conda_build_svn

Remove tag from local git and github

git tag -d V10-00-01 # removes tag on local git

git push --delete origin V10-00-01 # removes tag on github

List of commands to build psana1 release on s3df

Login and set environment

> s3dflogin

ssh psana -l psreldev

source /sdf/group/lcls/ds/ana/sw/conda1-v3/manage/bin/psconda.sh # conda1-v3 is a latest version of conda 23.10.0
conda deactivate
conda activate conda_build

Generate .tar.gz file with source code

cd /sdf/group/lcls/ds/ana/sw/conda1-v3/manage/ # if needed git clone git@github.com:slaclab/anarel-manage.git manage

bin/ana-rel-admin --force --cmd psana-conda-src --name 4.0.58 --basedir `pwd`

Build release

cd ~psreldev/git/psana1-feedstock/ # if needed git clone git@github.com:slac-lcls/psana1-feedstock.git

Update in recipe/meta.yaml fields for set version and sha256

conda build -c lcls-i -c conda-forge recipe

Debugging

In case of problem with tests look at log file like

/sdf/group/lcls/ds/ana/sw/conda1/inst/envs/conda_build2/conda-bld/psana_<13-digit-build number>/test_tmp/work/<log-file-name>

Upload .tar.gz file with release to anaconda lcls-i channel

anaconda upload -u lcls-i /sdf/group/lcls/ds/ana/sw/conda1/inst/envs/conda_build2/conda-bld/linux-64/psana-4.0.57-py39hed0727e_1.tar.bz2

Code Block

title	response on anaconda upload
collapse	true

(conda_build2) [psreldev@sdfiana001 psana1-feedstock]$ pwd
/sdf/home/p/psreldev/git/psana1-feedstock
(conda_build2) [psreldev@sdfiana001 psana1-feedstock]$ anaconda upload -u lcls-i /sdf/group/lcls/ds/ana/sw/conda1/inst/envs/conda_build2/conda-bld/linux-64/psana-4.0.57-py39hed0727e_1.tar.bz2
Using Anaconda API: https://api.anaconda.org
Using "lcls-i" as upload username
Processing "/sdf/group/lcls/ds/ana/sw/conda1/inst/envs/conda_build2/conda-bld/linux-64/psana-4.0.57-py39hed0727e_1.tar.bz2"
Detecting file type...
File type is "Conda"
Extracting conda attributes for upload
Creating package "psana"
Creating release "4.0.57"
The action you are performing requires authentication, please sign in:
Using Anaconda API: https://api.anaconda.org
Username: dubrovin
dubrovin's Password: 
login successful
Using Anaconda API: https://api.anaconda.org
Using "lcls-i" as upload username
Processing "/sdf/group/lcls/ds/ana/sw/conda1/inst/envs/conda_build2/conda-bld/linux-64/psana-4.0.57-py39hed0727e_1.tar.bz2"
Detecting file type...
File type is "Conda"
Extracting conda attributes for upload
Creating package "psana"
Creating release "4.0.57"
Uploading file "lcls-i/psana/4.0.57/linux-64/psana-4.0.57-py39hed0727e_1.tar.bz2"
15.4MB [00:01, 12.6MB/s]                                                                                                                                                                                  
Upload complete

conda located at:
  https://anaconda.org/lcls-i/psana

(conda_build2) [psreldev@sdfiana001 psana1-feedstock]$

Create new environment

conda create --name ana-4.0.58-py3 --clone ana-4.0.57-py3

Code Block

title	response on conda create
collapse	true

(base) [psreldev@sdfiana002 psana1-feedstock]$ conda create --name ana-4.0.58-py3 --clone ana-4.0.57-py3
Retrieving notices: ...working... done
Source:      /sdf/group/lcls/ds/ana/sw/conda1/inst/envs/ana-4.0.57-py3
Destination: /sdf/group/lcls/ds/ana/sw/conda1/inst/envs/ana-4.0.58-py3
Packages: 488
Files: 11

Downloading and Extracting Packages:

Downloading and Extracting Packages:

Preparing transaction: done
Verifying transaction: done
Executing transaction: /  
For Linux 64, Open MPI is built with CUDA awareness but this support is disabled by default.
To enable it, please set the environment variable OMPI_MCA_opal_cuda_support=true before
launching your MPI processes. Equivalently, you can set the MCA parameter in the command line:
mpiexec --mca opal_cuda_support 1 ...
 
In addition, the UCX support is also built but disabled by default.
To enable it, first install UCX (conda install -c conda-forge ucx). Then, set the environment
variables OMPI_MCA_pml="ucx" OMPI_MCA_osc="ucx" before launching your MPI processes.
Equivalently, you can set the MCA parameters in the command line:
mpiexec --mca pml ucx --mca osc ucx ...
Note that you might also need to set UCX_MEMTYPE_CACHE=n for CUDA awareness via UCX.
Please consult UCX's documentation for detail.
 
done
#
# To activate this environment, use
#
#     $ conda activate ana-4.0.58-py3
#
# To deactivate an active environment, use
#
#     $ conda deactivate

(base) [psreldev@sdfiana002 psana1-feedstock]$

conda deactivate
conda activate ana-4.0.58-py3
conda install -c lcls-i -c conda-forge psana=4.0.58

Code Block

title	response on conda install
collapse	true

(ana-4.0.58-py3) [psreldev@sdfiana002 psana1-feedstock]$ conda install -c lcls-i -c conda-forge psana=4.0.58
Channels:
 - lcls-i
 - conda-forge
 - defaults
 - lcls-ii
 - cogsci
Platform: linux-64
Collecting package metadata (repodata.json): done
Solving environment: done

## Package Plan ##

  environment location: /sdf/group/lcls/ds/ana/sw/conda1/inst/envs/ana-4.0.58-py3

  added / updated specs:
    - psana=4.0.58


The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    psana-4.0.58               |   py39hed0727e_1        15.4 MB  lcls-i
    ------------------------------------------------------------
                                           Total:        15.4 MB

The following packages will be UPDATED:

  psana                               4.0.57-py39hed0727e_1 --> 4.0.58-py39hed0727e_1 


Proceed ([y]/n)? y


Downloading and Extracting Packages:
                                                                                                                                                                                                    
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
(ana-4.0.58-py3) [psreldev@sdfiana002 psana1-feedstock]$

Create the same release on pcds

Info from Valerio

activate ana-4.0.58-py3 on S3DF,

>s3dflogin

ssh psana -l psreldev

source /sdf/group/lcls/ds/ana/sw/conda1/manage/bin/psconda.sh

conda deactivate

conda env list

conda activate ana-4.0.58-py3

then:

conda list --explicit > ana-4.0.58-py3.txt

copy ana-4.0.58-py3.txt to psreldev on PCDS - do this trick due to permissions:

>s3dflogin

cp ~psreldev/ana-4.0.61-py3.txt .

> pslogin

ssh psbuild-rhel7-01 -l psreldev

scp dubrovin@s3dflogin.slac.stanford.edu:ana-4.0.61-py3.txt . # complete copy of ana-4.0.61-py3.txt

source /cds/sw/ds/ana/conda1/manage/bin/psconda.sh

conda create -n ana-4.0.58-py3 --file ana-4.0.58-py3.txt

Code Block

title	response on conda create
collapse	true

(ana-4.0.54-py3) [psreldev@psbuild-rhel7-01 ~]$ conda create -n ana-4.0.58-py3 --file ana-4.0.58-py3.txt

Downloading and Extracting Packages
plotly-5.13.1        | ############################################################################################################################################################## | 100% 
gdk-pixbuf-2.42.10   | ############################################################################################################################################################## | 100% 
graphviz-7.1.0       | ############################################################################################################################################################## | 100% 
ipykernel-6.20.2     | ############################################################################################################################################################## | 100% 
psana-4.0.58         | ############################################################################################################################################################## | 100% 
Preparing transaction: done
Verifying transaction: done
Executing transaction: -  
For Linux 64, Open MPI is built with CUDA awareness but this support is disabled by default.
To enable it, please set the environment variable OMPI_MCA_opal_cuda_support=true before
launching your MPI processes. Equivalently, you can set the MCA parameter in the command line:
mpiexec --mca opal_cuda_support 1 ...
 
In addition, the UCX support is also built but disabled by default.
To enable it, first install UCX (conda install -c conda-forge ucx). Then, set the environment
variables OMPI_MCA_pml="ucx" OMPI_MCA_osc="ucx" before launching your MPI processes.
Equivalently, you can set the MCA parameters in the command line:
mpiexec --mca pml ucx --mca osc ucx ...
Note that you might also need to set UCX_MEMTYPE_CACHE=n for CUDA awareness via UCX.
Please consult UCX's documentation for detail.
 

done
(ana-4.0.54-py3) [psreldev@psbuild-rhel7-01 ~]$

Install new release on pcds:

conda install --experimental-solver=libmamba -c lcls-i -c conda-forge psana=4.0.58 # RECOMMENDED

conda install -c lcls-i -c conda-forge psana=4.0.58 # WORKING

edit and change release name in

pcds: /cds/sw/ds/ana/conda1/manage/bin/psconda.sh

s3df: /sdf/group/lcls/ds/ana/sw/conda1/manage/bin/psconda.sh > conda1-v3

Update release for Jupyter Notebook

Info from Chris:

ssh to pcds node > pslogin

ssh psbuild-rhel7-01 -l psreldev

Update version in TWO lines:

/cds/group/psdm/sw/conda/jhub_config/prod-rhel7/kernels/ana1-current-py3/kernel.json

Tags of packages included in the release

on pcds: /cds/sw/ds/ana/conda1/inst/envs/<environment>/data/anarelinfo/psana-conda-tags

on s3df: /sdf/group/lcls/ds/ana/sw/conda1/inst/envs/ana-4.0.62-py3/data/anarelinfo/psana-conda-tags

Detector conda_branch=False repo=psdm subdir=Non tag=V02-00-84 ?????

Extra operetions

See details in ~/.bash_history or ~/.k5login

cd /cds/sw/ds/ana/conda1/

if ./manage is not available

git clone git@github.com:slaclab/anarel-manage.git manage

cd manage

update it if necessary

git pull --rebase

conda create -n conda_build_py3 python=3.9 anaconda

conda env remove --name conda_build_py3

see /cds/sw/ds/ana/conda1/inst/envs/ OR conda info --envs

ssh psbuild-rhel7-01 -l psreldev

cd git/psana1-feedstock/

sha256sum /cds/sw/ds/ana/conda1/manage/downloads/anarel/psana-conda-4.0.56.tar.gz

> 443440bf7a7fdbcdf89314b68c07f97f5a89dff38d6da98685e74f0671fac37d

~psreldev/git/psana1-feedstock/recipe/meta.yaml

cd git
rm -rf psana1-feedstock/
git clone https://github.com/slac-lcls/psana1-feedstock

Set environment (chris):

source /cds/sw/ds/ana/conda2/manage/bin/psconda.sh. # Chris: conda2 because it needs py3, but conda1 also works with py3...
conda deactivate
conda activate conda_build

OR:

source /cds/sw/ds/ana/conda2/manage/bin/psconda.sh

conda create -n conda_build_py3 python conda-build anaconda-client

conda activate conda_build_py3

ssh psbuild-rhel7-01 -l psreldev

cd git
rm -rf psana1-feedstock/
git clone https://github.com/slac-lcls/psana1-feedstock

cp /cds/sw/ds/ana/conda1/manage/downloads/anarel/psana-conda-4.0.56.tar.gz /reg/g/psdm/web/swdoc/tutorials/

copy here because psreldev does not have permission !!!!???
because later it is used in psana1-feedstock/recipe/meta.yaml as url: https://pswww.slac.stanford.edu/swdoc/tutorials/{{ name }}-conda-{{ version }}.tar.gz

Issue: PermissionError: [Errno 13] Permission denied: '/cds/sw/ds/ana/conda2/inst/envs/conda_build/pkgs/libselinux-cos6-x86_64-2.0.94-h9b0a68f_1105'

Chris: cp ~cpo/.condarc ~/. # if ~/.condarc is not available

Valerio: conda create -n local_conda_build python conda-build conda-validate anaconda-client

scons test-Detector

git repository data_test_access cloned with all other psana packages
location for data_test can be controlled in config/psana-conda-svn-pkgs:
- subdir=/tmp - clones this package as /tmp/data_test
data_test_access is included but data_test is ignored in tar cfvz /cds/sw/ds/ana/conda1/manage/downloads/anarel/psana-conda-4.0.56.tar.gz psana-conda-4.0.56

Build on s3df

Initialization for build on s3df

IT IS ALREADY DONE ONCE

create conda_build_svn environment

s3dfnx - nx - server on s3df connects to dubrovin@sdflogin002

. /sdf/group/lcls/ds/ana/sw/conda1/manage/bin/psconda.sh

conda create -n conda_build_svn svn anaconda-client

clone anarel-manage

cd .../con-build OR cd /sdf/group/lcls/ds/ana/sw/conda1/
. /sdf/group/lcls/ds/ana/sw/conda1/manage/bin/psconda.sh

git clone git@github.com:slaclab/anarel-manage.git

make github seen from psana nodes

ema ~/.gitconfig

\[http\] - remove

Run bin/ana-rel-admin on s3df

login as myself

> s3dfnx ends up as dubrovin@sdflogin002

> psana ... because /afs/slac/g/pcds/svn/pdsdata is seen from psana nodes only... (Wilko)

cd <path-to>/anarel-manage

login as psreldev

> s3dfnx ends up as dubrovin@sdflogin002

ssh psana -l psreldev

cd /sdf/group/lcls/ds/ana/sw/conda1/manage

conda activate and run ana-rel-admin

source /sdf/group/lcls/ds/ana/sw/conda1/manage/bin/psconda.sh

conda deactivate

conda activate conda_build_svn

bin/ana-rel-admin --force --cmd psana-conda-src --name 4.0.56 --basedir `pwd`

Conda build

Everything works on s3df psana node under login psreldev because it needs some disk space... Other than that it can be built under any. other user.

Clone psana1-feedstock

IT IS ALREADY DONE ONCE for psreldev

> s3dflogin

ssh psana -l psreldev

cd git
rm -rf psana1-feedstock/
git clone https://github.com/slac-lcls/psana1-feedstock

Set environment

source /sdf/group/lcls/ds/ana/sw/conda1/manage/bin/psconda.sh

conda deactivate

conda activate conda_build_svn

Run conda build

cd ~/git/psana1-feedstock/

Update in ~psreldev/git/psana1-feedstock/recipe/meta.yaml fields for set version and sha256

...

References

Building Conda Packages And Releases
conda-cheatsheet.pdf
https://conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html#cloning-an-environment
/reg/g/psdm/etc/psconda.sh -> /cds/sw/ds/ana/conda1/manage/bin/psconda.sh
Running at S3DF
Convert svn repository to git
s3df-dynamic-sites-and-web-applications

Page tree

Page History

Versions Compared

Old Version 35

New Version Current

Key

Create new environment

Create new environment

Extra operetions

scons test-Detector

Build on s3df

Initialization for build on s3df

clone anarel-manage

make github seen from psana nodes

Run bin/ana-rel-admin on s3df

login as myself

login as psreldev

conda activate and run ana-rel-admin

conda1-v3

Conda build

Clone psana1-feedstock

Installing conda-build in the conda base

Set environment

Run conda build

Deprecated

pdsdata and psalg migrated from svn to github

create conda_build_svn environment

conda activate and run ana-rel-admin

Remove tag from local git and github

List of commands to build psana1 release on s3df

Login and set environment

Generate .tar.gz file with source code

Build release

Debugging

Upload .tar.gz file with release to anaconda lcls-i channel

Create new environment

Create the same release on pcds

Update release for Jupyter Notebook

Tags of packages included in the release

Extra operetions

scons test-Detector

Build on s3df

Initialization for build on s3df

create conda_build_svn environment

clone anarel-manage

make github seen from psana nodes

Run bin/ana-rel-admin on s3df

login as myself

login as psreldev

conda activate and run ana-rel-admin

Conda build

Clone psana1-feedstock

Set environment

Run conda build

References