Content

Psana-conda commands

TaskCommandNotes
Initialization of conda environmentsource /reg/g/psdm/bin/conda_setupBash only for conda, no .csh


Local release operations

Create local release directory

condarel --newrel --name <my-rel-directory>In current directory creates sub-directory <my-rel-directory> with a few system files
Activate local release

cd <my-rel-directory>

source conda_setup

 

Activate release in another directory

source conda_setup --reldir </path/to/another/release> 
Exit conda environmentundo_conda 

Upgrade release

conda activate ana-1.3.47

condarel --chenv

source conda_setup

scons -c

scons

In conda, first activate, using standard conda commands, the environment you want to build against.

Then use the --chenv command, it picks up the current conda environment.

Build code in release directorysconsBeing in local release directory


Package operations

Checkout package from master repositorycondarel --addpkg --name <pkg-name> --tag HEADCheck out latest version of package from github (or svn) repository where package resides.

Checkout existing package from psdm users repo

condarel --addpkg --user --name <pkg-name>

condarel takes --user flag to checkout from psdm users repo.

Create new package

condarel --newpkg --name <new-pkg-name>

use github GUI to create repository for this package.

Creates the directory <new-pkg-name> with a minimal

structure, include/src/app/data, and SConscript

Add pkg to psana build control

update the file psana-conda-svn-pkgs
in the directory
/reg/g/psdm/sw/conda/manage/config
that is part of the github repo slaclab/anarel-manage

This may be tricky due to permission to change code in slaclab/anarel-manage.

Chris or Clemens can do that.


Information commands

List available releasesconda env list 
Identify ana-currentmore /reg/g/psdm/sw/conda/current/ana/ana-current 
Identify dm-currentmore /reg/g/psdm/sw/conda/current/dm/dm-current 
Look at source code

ls /reg/g/psdm/sw/conda/scratch/<release>/<pkg>

ls $CONDA_PREFIX/lib/python2.7/site-packages/<pkg>

Package source code is available through the scratch directory.

In the conda environment only python code is available.


Example of regular development loop

Check in, work on package, update,

stage, commit, check out,

create new tag, etc.

condarel --newrel --name <my-rel-directory>

cd <my-rel-directory>

source conda_setup

condarel --addpkg --name <pkg-name> --tag HEAD

... edit files in the package

... run tests etc.

... add comments about changes

cd <pkg-name>

git pull --rebase

git status

git diff [module-name]

git add -A

git commit -m "comment" -a

git push origin master

git tag

git tag -a V01-00-26 -m "comment"

git push origin V01-00-26

Create local release directory (once) if it does not exist

move in release directory

set and activate conda environment

add package(s) if necessary

work on code

 

add comments in file <pkg-name>/doc/ChangeLog

when ready to commit

update package from repo, if someone else may work on it

print info about package status

see difference in modules

stage modified modules under git control

commit changes to git

commit changes to github master repo

print available tags

create new tag

commit new tag to github repo


Special cases

develop pdsdata/psalg

condarel --addpkg --name pdsdata

condarel --addpkg --name pdsdata_ext

See also Building the psalg and pdsdata packages
With conda, pdsdata and psalg are part of the
psana-conda package. They get put in a subdirectory
called extpkgs. You need the proxy packages to
build/develop.
develop ndarray

condarel --addpkg --name ndarray --tag HEAD
condarel --addpkg --name ndarray_ext
scons 
tag ndarray when done

update version in ndarray recipe meta.yaml:
  conda package version
  git url -- your new tag
build new ndarray package (admin account):
  cd /reg/g/psdm/sw/conda/manage
  git pull (or fetch? Get you edits above)
  cd recipes/psana
  ana-rel-admin --cmd bld-pkg ndarray

ndarray live in it's own conda package.
The ndarray github repo does not include a SConscript.
You need the ndarray_ext to link it into the build system.
If ndarray_ext sees you've checked out ndarray, it overrides
the ndarray in the conda environment.

After tagging your changes to ndarray, a new ndarray conda
package must be built. Edit the meta.yaml on github in the
recipe in the links to the right. Now from the admin account,
update the management code in
/reg/g/psdm/sw/conda/manage.
Now build a new ndarray package with ana-rel-admin.
The next time a release is built, the new ndarray is picked up.

 

References

  • No labels