SCons A Demo Implementation using SciTools

Introduction 

Navid has been busy setting up a demonstration environment using SCons and the ScienceTools v8r2.

Location of Installation

SCons is installed here at SLAC:  /afs/slac/g/glast/applications/SCons/0.97/bin/scons
There's a copy of SciTools in Navid's area:  /nfs/farm/g/glast/u06/golpa/ST-v8r2-scons/ScienceTools
If you desire to run SCons, please copy the ScienceTools from Navid's area into a place you have write-access.  Once you have a copy, enter the top level of the ScienceTools directory where the SConstruct file is located and run:

scons -h

This will provide a list of available options as well as how to specify compile options such as debug and optimize.  If you desire to build you would just type:

scons

Interesting Details and things we may need to discuss

There is a top-level SConstruct file which contains configuration/build information to be used "globally".  This includes options for the compilation such as debug and optimization flags and tells SCons the list of packages associated with this project and how to find the SConscript files for each package.  See the Gory Details of the SConstruct file for more information.  There is another file at the top-level called externals.scons which is used to set up the location of the external libraries.  More on this point further down in the External Libraries section.

Location of SCons specific files and package structure

Each package contains its own SConscript file which act as SCons' version of CMT's requirements file.  You can see an example for the facilities package further down on this page.  The SConstruct file communicates the location of the SConscript file for each package.  We may wish to consider storing the SConscript file in a directory similar to our current cmt directory.

The virjpk subdirectories are not in place or assumed in this organization.  There is a note out to Riccardo to check if that arrangement is acceptable to MRStudio.  Doing without the package-tag subdirectories allows us to utilize CVS directly for our checkouts, rather than devising a way to recreate CMT checkouts.  Using this organization, users would obtain code and build via the following commands:
cvs co -r v8r2 ScienceTools
cd ScienceTools
scons

External Libraries

By default, SCons provides a mechanism similar to automake that allows users to specify the location of external libraries and their headers using --with-blah-lib and --with-blah-include 

Navid is concerning adding an option for GLAST_EXT that would allow a user to specify a top-level GLAST_EXT directory structure, similiar to what what we have now. 

The scons-specific file:  externals.scons sets up the external libraries and adds the appropriate -L options to the baseEnv and specifies the specific libraries from the externals to include in a build.  Currently this externals.scons file is devoid of any version information, it is currently left to the user to provide the appropriate paths to the external libraries they desire to build against. This clearly allows more flexibility on the part of users.  Using this paradigm we'll have to be careful to provide specific instructions in the workbook to point to our recommended versions and point out any gotchas such as ROOT being built against a specific version of python for instance in helping users determine what may be safe in terms of providing a user-defined version of some of the externals.

 Building Against A Release

SCons  will not accept having two versions of the same library - so we must devise a method to over-ride the one located in the shared release.  A potential example:

A local version of package X is installed and we wish to build against a release stored at SLAC on u30.  We start a scons build pointing at the SConstruct file located on u30 with an override directory pointing at the local copy of the package you want to override.  The SCons compile would proceed for your local package pointing at the includes and libraries in u30.

Gory Details of the SConstruct file

The SConstruct file begins by setting up the global environment for the build.  This is done by updating SCons' environment variable "baseEnv".  Here are the contents of the SConstruct file:

import os,platform
#########################
#   Global Environment  #
#########################
baseEnv = Environment();
if ARGUMENTS.get('debug',0):
        baseEnv.Append(CCFLAGS = "-g")
if ARGUMENTS.get('optimized',0):
        baseEnv.Append(CCFLAGS = "-O2")
if ARGUMENTS.get('CC',0):
        baseEnv.Replace(CC = ARGUMENTS.get('CC'))
if ARGUMENTS.get('CXX',0):
        baseEnv.Replace(CXX = ARGUMENTS.get('CXX'))
if ARGUMENTS.get('CCFLAGS',0):
        baseEnv.Append(CCFLAGS = ARGUMENTS.get('CCFLAGS'))
if ARGUMENTS.get('CXXFLAGS',0):
        baseEnv.Append(CXXFLAGS = ARGUMENTS.get('CXXFLAGS'))
helpString = """
Usage:

        scons [target] [compile options]

Compile Options:
        Optimized:              Set to 1 to compile with optimization. Default 0.
        CC:                     Set to the compiler to use for C source code.
        CXX:                    Set to the compiler to use for C++ source code.
        CCFLAGS:                Set to additional flags passed to the C compiler.
        CXXFLAGS:               Set to additional flags passed to the C++ compiler.
"""
Export('baseEnv')

The next section pertains to externals:

#########################
#  External Libraries   #
#########################
helpString += SConscript('externals.scons')

Help(helpString)

The following sets up the directories which will contain the libraries, binaries, and include files: 

#########################
#  Project Environment  #
#########################
libDir = os.path.join(os.path.abspath('.'),'lib')
binDir = os.path.join(os.path.abspath('.'),'bin')
incDir = os.path.join(os.path.abspath('.'),'include')
testDir = binDir

baseEnv.Append(LIBPATH = [libDir])

Export('libDir','binDir','incDir','testDir')

Create a list of packages for this SCons project and define the registerObject function which will be used by each package in their respective SConscript file to define the list of libraries, applications, public headers, etc that are desired for the global compilation.

packages = [

def registerObjects(package, objects):

        libs = []

Export('registerObjects')

Lastly we tell SCons where to find the SConscript file for each package:

for pkg in packages:
        SConscript(os.path.join(pkg,"SConscript"))

An Example SConscript File

Here is the SConscript file for the facilities package.  Each package will have its own SConscript file, similar to the requirements file from CMT: 

import glob,os

Import('baseEnv')
Import('registerObjects')
env = baseEnv.Copy()

facilitiesLib = env.StaticLibrary('facilities', glob.glob(os.path.join('src','*.cxx')))

This line creates a variable called facilitiesList.  It says to create a static library called facilities and these are the source files
<span style="color: #3300ff">registerObjects('facilities',</span> { 'libraries': [facilitiesLib], 'includes': glob.glob(os.path.join('facilities','*.h'))}

)

We set up the compiler options, macros, etc in a SCons "environment".  In this case, the environment is imported from "baseEnv" which is set up in the SConstruct and the external.scons files at the top-level of ScienceTools.  It is imperative that the SConscript files COPY the baseEnv, and update the copy, rather than updating baseEnv directly.  The reason is that any changes to baseEnv will propagate to all packages, even retroactively.

The registerObjects function is called to setup the items we wish to submit from facilities to be part of the global build.  This is all just preparation, no building it done yet, we are just storing the information needed by scons to perform the build.  The first argument is what the package is called, in this case, "facilities".  The second argument is a hash of arrays.  The hash name is the type of object you are registering (libraries, include files, binaries, test apps, etc.), and the value of the hash is a list of those objects.

After all the SConscript files have been called, and registerObjects updated for each package, scons is ready to compute the dependencies and start the build in the correct order.

Interesting Python filesystem manipulation routines

os.path.join joins 'src' and '*.cxx' together in the correct way ie \ for windows and / for linux
glob.glob then takes the src/*.cxx and gets a list of files that match that pattern