• Created by Unknown User (davidsch), last modified on Dec 12, 2016

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 6 Next »

Introduction

This page covers documentation about the conda release and build system relevant for psana developers. This documentation is also for LCLS users that want to migrate their test releases based on psana C++ modules and the SConsTools scons build system to the conda environments.

Old to New Conda Command Table

Below we go over the steps to create, develop, build and manage test releases - comparing old commands from the RPM release system to the new conda based system.

Old commands like newrel, addpkg and relinfo are not available in the conda world. Most all steps are executed using the

condarel

program. Do condarel -h for the latest help on this script.

stepoldcondanotes about conda version
get startedsource
/reg/g/psdm/etc/ana_env.sh

source
/reg/g/psdm/etc/ana_env.sh

source conda_setup

We still source the ana_env.sh to get going
After sourcing conda_setup,
/reg/g/psdm/bin is removed from your PATH
it is replaced with
/reg/g/psdm/sw/conda/manage/bin
create new test
release directory		newrel ana-current myrelcondarel --newrel --name myrel

in conda, myrel is based on the current
conda environment. The previous conda_setup
command activated a conda environment like
ana-1.0.7.

If you are not in a conda environment with
psana-conda installed, condarel will fail.
(sourcing conda_setup will automatically
activate such an environment)

The old myrel directory has the hidden file
.sit_release with content like ana-0.19.21
In the new myrelease directory, the content will be
the psana-conda package name and version.

The new myrel directory will also have the
hidden file .sit_conda_env with the full path of the
conda environment myrelease is built against

activate
test release		cd myrel
sit_setup		cd myrel
source conda_setup

conda_setup looks in the current directory (like
sit_setup) for the special files mentioned
above. It sets PATH, LD_LIBRARY_PATH and
PYTHONPATH to first look for programs,
libraries and python modules built in your
test release before looking for them in the conda
environment.

conda_setup will add *tr* to your prompt to
indicate that you are in  a test release.

create
new package		newpkg MyPkgcondarel --newpkg --name MyPkgCreates the directory MyPkg with a minimal
structure, include/src/app/data, and SConscript
create
new package
in psdm svn
repo		psvn newpkg MyPkgsamenothing created in your test release, just
starting a package in the svn psdm repo
checkout new
package		addpkg MyPkgcondarel --addpkg --name MyPkgfor packages that are not part of psana-conda
gets it from head
checkout
existing package		addpkg XtcInputcondarel --addpkg --name XtcInputLooks up the appropriate tag for the version of
psana-conda. Checks out that tag
checkout from
HEAD		addpkg XtcInput HEADcondarel --addpkg \
--name XtcInput --tag HEAD		You can also specify specific tags with the
--tag argument.
buildsconssconssame

develop
pdsdata/psalg
or
ndarray

very awkwardcondarel --addpkg --name pdsdata
condarel --addpkg --name pdsdata_ext		With conda, pdsdata, psalg and ndarray are part of the
psana-conda package. They get put in a
subdirectory called extpkgs. You then also need
the proxy packages to build. Note - the pdsdata_ext
SConscript make command does not generate the ddl
targets - you may need to set environment variables
and run make yourself for certain development.
release
info		relinfocondarel --relinfo 
upgrade
release
relupgrade ana-19.0.20
sit_setup
scons -c
scons

source activate ana-1.0.8
condarel --chenv
source conda_setup
scons -c
scons

In conda, first activate, using standard conda
commands, the environment you want to build against.
Then use the --chenv command, it picks up the
current conda environment.
develop
Translator		addpkg Translator
scons

condarel --addpkg --name Translator
condarel --addpkg --name hdf5
condarel --addpkg --name openmpi
scons

Since the Translator includes headers using package
names, i.e,
#include "hdf5/hdf5.h"
You must first include the hdf5 and openmpi proxy
packages
Testscons testscons test 
tag finished
work for
psdm repo		addpkg MyPkg HEAD

cd MyPkg
svn update
psvn tags
psvn tag V00-00-00

condarel --addpkg --name MyPkg
--tag HEAD
...
same
...

the psvn program functions as before, you can
run psvn tags to see all the tags, then psvn tag
to create a new tag.

Missing Functionality

Not all functionality of addpkg, relinfo, sit_setup etc have been implemented. If there is a feature that you need, let me know.

If you want to use the old commands, i.e, addpkg instead of condrel --addpkg, we can write new wrappers - but I think while we transition it is good to keep the interfaces distinct as psana developers will be working with both build systems.

Converting a Release

I recommend leaving an old release alone and starting new ones based on conda, however there are two commands in condarel to convert back and forth. This should work for users writing their own C++ psana modules, but won't work for psana developers that have checked out certain external proxy packages. These commands are

condarel --convert2conda

note the name of the old rpm release that you lost. If you want to switch back, use that name with the

condarel --convert2rpm

command. See condarel -h for details.

 

 

 

  • No labels