Introduction

Below is some more detailed/advanced information about the offline build system. The offline build system is based on SCons. It is implemented as a set of scripts that import functionality from SConsTools (software developed internally). SConsTools introduces a package and release structure to support the build process.  The top level directory is for the release and contains the SConstruct file. Each package is in a sub-directory to the release.  Packages have a SConscript file. The structure of a release directory is explained in this page about Packages and Releases. SConsTools has a number of features including:

...

release/include
release/build
release/arch

You may also see

release/data
release/web

depending on the packages being built. These directories belong to the build system - you should never put anything in these directories. They will be rebuilt each time scons runs if need be.
In release/include you will find two links to the package include directories:

...

In the arch directory, you will find all the executables, scripts, libaries libraries (built as shared object libraries) and python modules for each package.

...

Most packages require no additional options beyond those in the default configuration. If a package requires additional build options, these can often be added by calling the standardSConscript() function in the SConscript file in the package directory. For instance, suppose a psana - Original Documentation user is developing a module in a package called MyPackage which needs to use functions from the Gnu Scientific Library. They would add the following line:

...

While targets in packageA that use mylib can now be built correctly, to run them you will need to make sure that the operating system can find the shared object library mylib.  This is done by setting the environment variable LD_LIBRARY_PATH to include the directory where mylib is located. This could be done in the shell initialization file (such as .bashrc, or .cshrc, depending on which shell you use). 

Using a Library through the External Package Mechanism

What if you want to make an external library available to all the packages in your release? It would be tedious to modify each new package you develop as in the section above. A better solution is to create an external package that interfaces to the library. In your release directory, do the following:

newpkg MyCode

edit the file

~/release/MyCode/SConscript

This SConscript calls the funciton standardSConscript(), but this is not what we want. We want to use standardExternalPackage(). Change the file to look like:

# Do not delete following line, it must be present in 
# SConscript file for any SIT project
Import('*')

import os
from SConsTools.standardExternalPackage import standardExternalPackage
#
# For the standard external packages which contain includes, libraries, 
# and applications it is usually sufficient to call standardExternalPackage()
# giving some or all parameters.
#
PREFIX = os.path.expanduser('~username/mycode')
INCDIR = "include"
LIBDIR = "lib"
PKGLIBS = "mylib"
standardExternalPackage('MyCode', **locals())

This is the only file that need by in the MyCode package. After doing scons, any package in the release will be able to call functions in mycode by doing

#include "MyCode/myheader.h"

The build system will identify the dependency on the MyCode package. Given the PKGLIBS parameter, it will then add mylib to the link line. libmylib.so will be found because PREFIX and LIBDIR together give the directory where the MyCode shared object libraries are. This directory is not added to the LD_LOAD_LIBRARY path or explicitly used when building. By default, standardExternalPackage() will make soft links from the $SIT_ARCH/lib directory of your release to all the shared object libraries in the external package lib directory. When you ran sit_setup from your release directory, it placed the $SIT_ARCH/lib directory of your release first in the $LD_LIBRARY path. This is the mechanism by which MyCode libraries are made available to all packages in your release.

The above SConscript file for an external package is the simplest case. MyCode has a common directory structure: a base directory with an include and lib directory as sub-directories. Thus the PREFIX, INCDIR and LIBDIR parameters work well to describe it to SConsTools. If the include and lib directory were in different places, one could omit the PREFIX parameter and give absolute paths for INCDIR and LIBDIR:

INCDIR = os.path.expanduser('~username/mycode/include')
LIBDIR = os.path.expanduser('~username/mycode/lib')

Many external packages are more than just a library. They have stand alone programs or tools to run. You can add these using the BINDIR parameter.  If an external package like MyCode were to in turn depend on another external package, that package could be defined in the same SConScript file and the DEPS parameter could be used when defining MyCode with standardExternalPackage. An example of using these options can be seen in SConscript file for the hdf5 package. One can do

addpkg hdf5

Within your release and take a look at hdf5/SConscript.Need to add this section

Building Part of the Release

...

To upgrade the version of a release, see Typical Common development tasks.

Presentations

Below are slides from a talk given during the 2014 LCSL users meeting that covers the release/package structure of SConsTools, as well as how psana modules: Software_Development_Environment.pdf