Introduction

This sections lists several typical tasks that users and developers will perform frequently. It implies that the SIT environment has already been setup as explained in Environment setup.

Our standard development machine is psdev, all commands below are expected to work from this machine. Analysis farm machines should not be used for development.

Currently our repository is setup on AFS, to access it we use ssh tunnel to yakut machines. Before running SVN commands make sure that you can connect from psdev to yakut with ssh – try to run ssh yakut. To avoid password prompts from svn commands you may need to run kinit on psdev to obtain AFS token. SVN repository is accessible to accounts in a special AFS group (g-lusi), if you are not a member of this group ask Andy or Igor to add you account to this group.

Tasks

Using different release versions

User release directories involve a mix of packages from the user and system. scons will make sure the version associated with each is the same. Steps to upgrade a user release are below. When just using the system release - for example running psana from the home directory, you can easily switch the release version as follows:

To switch to "newest" release:

sit_setup ana-current

To switch to a numbered release and use debug build:

sit_setup 1.2.3 dbg

Several environment variables that start with SIT_ will be changed by the above commands.

Switching to a different release or build options

For a user release directory, you may want to switch build options, or upgrade the release directory.

To use debug build

sit_setup dbg

Switch to other release permanently

relupgrade <new-release-name>
sit_setup
scons -c
scons

Creating test release based on some numbered release

All available releases can be found in the directory SIT_RELDIR

cd <test-area>
newrel 1.2.3 test-1.2.3
cd test-1.2.3
sit_setup

Check out package from repository

To check out package HEAD

addpkg MyPackage HEAD

To check out the same tag as in the release

addpkg MyPackage

To check out specific tag

addpkg MyPackage V00-00-00

Check the status of the files in package

Run this command often to see if you forgot to add any local files to repository

svn status MyPackage

this command shows the local status of the package with respect to the last svn commit (check out) command.

svn status -u (or --show-updates)

shows files which will be updated if you run svn update command.
See more info in svnbook.

Check tags of local packages

The tags of packages located in the release directory can be listed by the command

relinfo

The tags of packages of any release can be listed by the command

relinfo <release-name>

How to see which new tags were added on top of the specified release (for example ana-0.13.3)

kinit

relinfo -n -P -f /reg/g/psdm/sw/releases/buildbot/tags/tags-ana REPO > latest-tags

diff latest-tags /reg/g/psdm/sw/releases/buildbot/tags/ana-0.13.3

Create completely new package

This will create basic structure for a regular package - package directory, SConscript file, doc/README, and doc/ChangeLog.

newpkg MyNewPackage

This command does not change anything in the repository, it only creates local directory and files.

Create new package in Subversion

psvn newpkg MyNewPackage

and check it out

addpkg MyNewPackage

Remove package from release directory

In order to get rid of traces of the package in the release directory, all binary files need to be cleaned by the command scons -c before removing the package. For example,

cd <release-directory>
sit_setup
scons -c
rm -rf <package-name>

Add files to repository

svn add <file-or-dir> ...

If the argument is a directory then all files in that directory will be added too.

Display modifications to a package

Display all local modification in a working copy

svn diff <package>

Display all local modification to a particular file

svn diff <package>/<file>

Display diffs between local copy and the HEAD of the package in repository

svn diff -r HEAD <package>

Display changes happened in repository between time when the package was checked out and now:

svn diff -r BASE:HEAD <package or path to module>

where

• BASE is the revision of the package when it was checked out
• HEAD is the latest revision of the same directory in SVN

For example

svn diff -r 7810:HEAD ParametersV1.h

To see ALL modifications made to a <package-name> since the specified SVN revision (ex.: 8968)

svn diff -r 8968:HEAD https://pswww.slac.stanford.edu/svn-readonly/psdmrepo/<package-name>/trunk

For more complex cases consult SVN documentation.

Building the release

scons

or

scons TRACE=1

Use higher TRACE numbers for verbose output.

To compile all unit tests in the release

scons test

or to build test for particular package:

scons test <package>

Remove or rename files

svn rm <filename>

svn mv <filename> <new-filename>

Committing changes to a package

svn status <package>
svn commit -m "Log message for this commit" <package>

List existing tags for a package

cd <package>
psvn tags

or

psvn tags <package>

Creating new tag for a package

Before you create new tag run 'svn update' command:

cd <package>
svn update
psvn tag V01-02-03

or

svn update <package>
psvn tag -p <package> V01-02-03

Problems can arise if your working directory is checked out from a tagged release in the repository (you cannot make changes into something that is tagged - it has been finalized). If this is the case, do the following

svn info #this prints the URL you have checked the package out from, is /tags/ in the path?
svn switch <path to to trunk location for the package>
svn update # get the latest from trunk
svn ci ... # commit any local changes you wish to trunk, update doc/ChangeLog in your commit
psvn tag V01-02-03

If the psvn tag command doesn't work, it may be because your are having svn copy code from the working directory rather than from a revision of trunk into a tag. The command

psvn tag V00-28-08

is equivalent to

svn copy . $SIT_SVN/package/tags/V00-28-08

which probably does not do exactly what you intend. TO fix this you need to copy trunk to tags. This can be done in couple of ways:

1. Find revision of the files that you committed (appears in output of svn commit, say it is 6404), then use it with psvn tag:

psvn tag -e 6404 V00-28-08

2. Use svn copy:

svn copy $SIT_SVN/package/trunk $SIT_SVN/package/tags/V00-28-08

Package sub-directory names convention

Command

newpkg MyNewPackage

creates the sub-directory tree with a minimal set of directories/files in it. For most projects the list of sub-directories need to be extended. For example,
for C++ projects it is convenient to add sub-directories with pre-defined names:

cd <your-release-directory>
mkdir <package>/include
mkdir <package>/src
mkdir <package>/app
mkdir <package>/test

for C++ header (.h), source (.cpp), application (.cpp), and test-application (.cpp) files, respectively.
By default all files in src with include will be compiled and put in the package library. The files in app will be also compiled and moved in the <release>/arch/<architecture>/bin - default release bin directory. In order to compile and run modules from test directory use commands:

scons test

or

scons test <package>

then, to run:

build/<architecture>/<package>/<test-module>

These modules will not be moved to the release bin directory.

Updating svn property variables

Every module under svn control of may heve/use associated property variables, which may be updated at svn commit command. The default  property variables are listed in table:

Usage of these variables should be allowed in ~/.subversion/config file,

To update all keywards in the module before commiting use command:

psvn mktxtprop <path-to-modeule(s)>

which is a wrapper on svn propset command. To update revision only, use command:

svn propset svn:keywords "Revision" <path-to-modeule(s)>

Adding package in release build

For RPM-based build - edit the file ana-tags in the directory /reg/g/psdm/sw/releases/buildbot/tags/

For conda-based build - update the file psana-conda-svn-pkgs in the directory /reg/g/psdm/sw/conda/manage/config/

Examples

Create a new package

This example shows how to (1) create a new release, (2) to create new package, (3) add directories and modules, (4) commit to svn, and (5) set a tag.

ssh psdev
cd <directory-with-your-favorite-releases>
newrel ana-current <your-release-directory>
cd <your-release-directory>
sit_setup

# create local package
newpkg <new-package>

mkdir <new-package>/src
codegen -l pyana-module <new-package> <module1>
   or
cd <new-package>
mkdir <new-directory>
cp <path>/<module2> <new-directory>/<module2>
...
edit <module1> <module2>
...

# create package in SVN and check it out
psvn newpkg <new-package>
addpkg <new-package>

# add files to repository
svn add <new-directory>
   or
svn add <new-directory>/<module1>
svn add <new-directory>/<module2>

svn commit -m "Log message for this commit" <new-package>
psvn -p <new-package> tag V01-00-01

Edit existing package

ssh psdev
cd <directory-with-your-favorite-releases>
newrel ana-current <your-release-directory>  # or use already existing release directory
cd <your-release-directory>

sit_setup            # To use the same release
   or
sit_setup newest     # To switch to "newest" release
   or
sit_setup 1.2.3 dbg  # To switch to a numbered release and use debug build

addpkg <existing-package> HEAD       # To check out package HEAD (latest version)
cd <existing-package>
svn update

   [Edit, add remove modules and directories, build ({{scons}}), and test applications, etc...]

svn commit -m "Log message for this commit"
svn update                           # To account for possible modifications from other developers
psvn tag V00-00-06                   # Incremented package version tag