Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Migrated to Confluence 4.0

...

/
    SConstruct
    externals.scons            List of externals and their versions.  Can utilize GLAST_EXT or over-ride individual libraries.  Check scons -h for available options.
    (Likelihood)                  Directories are in ( )
    (facilities)
        SConscript
        facilitiesLib.py            <package>Lib.py lives in each package
    (site_scons)
        (site_tool)                  SCons looks here by default for tools
    (include)                       installation location for public headers
    (lib)                              installation location for libraries

automated dependencies

Recommended fix from SCons developers is to introduce a SCons tool (a python function) which is used to update the environment.  Tools are located in the site_tool directory by default, this can be over-ridden via an option when invoking the tool.  Rather than use the site_tool location, there is a <package>Lib.py file within each package.  Similar in function to CMT's "use" statements, this file lists what other packages this package requires.  Only lists direct dependencies, SCons figures out the rest.

...

Python imports:
import os, glob
Import('baseEnv')     Note the big "I" for a SCons import.  Also note that the objects are writable and changes will propogate to all.
Import('registerObjects'

Navid is looking for a way to make the objects read-only, but for now, we make a copy of the baseEnv so that clients will not modify it and muddy it for others.

...

Now to define the facilities static library, similiarly for a shared library.
facilitiesLib = env.StaticLibrary('facilities',                                               library name
                                             glob.glob(os.path.join('src', '.cxx'))  *       glob.glob explands wildcards  os.path.join inserts the appropriate slashes for each OS.
We can utilize subdirectories for the source location as well.

Wiki Markup*{_}registerObjects('facilities', {'libraries':\[facilitiesLib\], 'includes':\[glob.glob(os.path.join('facilities', '\*_{*}_._{*}{_}h'))_*

Each package contains a tool,  <package>Lib.py which defines a python function according to SCons' specific form.  The facilitiesLib.py file contains two functions:
generate which does the work, and exists which provides the ability to turn off the tool.

def generate(env, **kw):  dictionary of keywords, which is not used now for ST where single libraries are created but GR will create packages where libraries may have different dependencies.

Wiki Markupenv.Tool('addLibrary', library=\['facilities'\])&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; to call a tool&nbsp;&nbsp;&nbsp; addLibrary was created by Navid.&nbsp; This adds the facilities to a list of         to call a tool    addLibrary was created by Navid.  This adds the facilities to a list of libraries.

site_tools contains addLibrary
ordering in gcc matters for libs - can't find symbols otherwise.  The addLibrary tool allows developers to avoid worrying about the order themselves, since the tool reorders as necessary.   addLibrary makes sure the order is correct, if the item is not already the list, add it to the end, if it is already there, move it to the end.  Each library only appears once - this helps reduce the chance of creating a command line that is too long (which can occur when dealing with the Gaudi libraries).

Discussion

Wiki MarkupAfter building the libraries will exist in both the packages and in the installation lib directory.&nbsp;   After the build is completed, all we need is the include and lib directories for runtime. _\
[Actually this was true at the time of the discussion, but since then Navid has backed off having the SCons build perform the installation step and will copy the necessary files to the installation directories as a separate step.&nbsp;   Aug 20, 2007\]_

Jim reminded us that it would be helpful to provide a mechanism for tags such as rh9_gcc32 and rhel4, similarly for opt and debug builds.  Navid could implement subdirectories with expanded English names as he's found some users are confused by our use of rh9_gcc32 for instance, something like redhat9_optimized.

Toby made a request that the new RM avoid "big bang builds" which take oodles of time.  He suggested that packages common to both ST and GR could be pulled into their own checkout package.  Navid stated that the SCons builds are much faster than the CMT builds.

Visual Studio 2005 - Toby is hoping Riccardo can fix up MRvcmt to use his python scripts to work with VS2005.  Or perhaps MRvcmt is considered frozen - we need to discuss this with Riccardo as soon as possiblee. 

Defining Applications using Likelihood as an example

...

Import('baseEnv')
Import('registerObjects')
env = baseEnv.Copy()
env.Tool('LikelihoodLib')            The applications can depend on the Likelihood library, LikelihoodLib.py sets up the environment to handle this.unmigrated-wiki-markup

*{_}LikelihoodLib = env.StaticLibrary('Likelihood', \ [glob.glob(os.path.join('src','.c')), glob.glob(os.path.join('src','.cxx'))\])_* *{_}gtlikelihoodBin =
gtlikelihoodBin = env.Program('gtlikelihood',glob.glob(os.path.join('src','likelihood','_{*}_.cxx')))_\*

The contents of LikelihoodLib.py:

Wiki Markup&nbsp;*{_}def  def generate(env, \ **kw):_* *_&nbsp;&nbsp;&nbsp;
    env.Tool('addLibrary', library=\['Likelihood'\])_* *_&nbsp;&nbsp;&nbsp;
    env.Tool('astroLib')_* *_&nbsp;&nbsp;&nbsp;
    env.Tool('xmlBaseLib')_* *_&nbsp;&nbsp;&nbsp;
    env.Tool('tipLib')_* *_&nbsp;&nbsp;&nbsp;
    env.Tool('evtbinLib')&nbsp;&nbsp; &nbsp;&nbsp; &nbsp;&nbsp; &nbsp;&nbsp; &nbsp;&nbsp; &nbsp;&nbsp; &nbsp;&nbsp; &nbsp;&nbsp; &nbsp;&nbsp; &nbsp;&nbsp; &nbsp;&nbsp; &nbsp;&nbsp; &nbsp;&nbsp; &nbsp;&nbsp; &nbsp;&nbsp; &nbsp;&nbsp; &nbsp;&nbsp; &nbsp;&nbsp; &nbsp;&nbsp; &nbsp;&nbsp; &nbsp;&nbsp; &nbsp;&nbsp;_* <span style="color: #3300ff"><strong>The libraries can be added in any order, they will be sorted later</strong></span> *_&nbsp;&nbsp;&nbsp;                                                                   The libraries can be added in any order, they will be sorted later
    env.Tool('map_toolsLib')_* *_&nbsp;&nbsp;&nbsp;
    env.Tool('optimizersLib')_* *_&nbsp;&nbsp;&nbsp;
    env.Tool('optimizersLib')_* *_&nbsp;&nbsp;&nbsp;
    env.Tool('irfLoaderLib')_* *_&nbsp;&nbsp;&nbsp;
    env.Tool('st_facilitiesLib')_* *_&nbsp;&nbsp;&nbsp;
    env.Tool('dataSubselectorLib')_* *_&nbsp;&nbsp;&nbsp;
    env.Tool('hoopsLib')_* *_&nbsp;&nbsp;&nbsp;
    env.Tool('st_appLib')_* *_&nbsp;&nbsp;&nbsp;
    env.Tool('st_graphLib')_* *_&nbsp;&nbsp;&nbsp;
    env.Tool('addLibrary', library=env\['cfitsioLibs'\])&nbsp;&nbsp; &nbsp;&nbsp; &nbsp;&nbsp; &nbsp;&nbsp; &nbsp;&nbsp; &nbsp;&nbsp; &nbsp;&nbsp;_* <span style="color: #3300ff"><strong>Note how the external libraries are handled</strong></span> *_&nbsp;&nbsp;&nbsp; ])                     Note how the external libraries are handled
    env.Tool('addLibrary', library=env\['cppunitLibs'\])_* *_&nbsp;&nbsp;&nbsp;
    env.Tool('addLibrary', library=env\['fftwLibs'\])_*

 Navid also mentioned that ROOT libraries are setup similarly to how they are handled now with a standard set of RootLibs, and a set of GUI libs.

Issues

By default, SCons searchs for tools in the site_tools directory, or the user can provide a list of tool paths.  This mechanism is working except in the case of a container package such as celestialSources.  Navid is trying to avoid hard-coding anything into the SConscript files and external tools.  The worst case scenario is that a container will have to announce itself as such.

Private includes should remain in the src directory as suggested currently.  Actually, there were some public includes in some of the ST package, such as optimizers.  Jim will look into that.

Navid is working on overriding package, aka multiple CMTPATHs. 

We spoke a little about alternatives to the SCons' suggestion of automating the locating of dependencies.  Jim had a method where a new environment was defined for each package.  One could them import that environment and the appropriate dependencies.  Some potential draw backs is that the ordering of dependencies does matter and there is a problem when working with GR and Gaudi in that the command line can get too long.  Navid's method avoids those issues.

At runtime, after a build, SCons is no longer necessary.

We still need to address our use of environment variables such as <package>ROOT.  Navid's commonUtilities class was created to allow pacakges to register their environment variables and perform the setenv and getenv calls.  There will be a handful of remaining environment variables that must be defined a head of time, such as the location of the calibration files. 

Navid's ultimate goal is to create the mechanism for SCons to build and generate files for the IDE of the developer's choice, including Visual Studio and KDE.

CVS Layout Discussion

A Request From Our Friends at the GSSC

Eric Winter, on behalf of the Science Support Center, asked about the future of the CMT requirements files.  Currently hmake reads in the requirements files to set up its builds of the science tools.  We plan to keep the CMT requirements file for about six months, as we continue to prepare for our move to SCons.  We will have to provide some replacement, some SCons tool, for the GSSC to use instead.  Currently, the SConscript and <package>Lib.py files contain the same information as the CMT requirements files.  Brian Irby is man in charge of hmake. 

To Do List

  • Finish demonstration of SCons using ScienceTools, including over-riding packaegs / multiple CMTPATHs
  • Windows and Visual Studio
  • Use commonUtilities to replace our current use of environment variables
  • Create ScienceTools-scons in CVS
  • Create a new RM which is separate from our current one.
  • Determine the schedule for MRStudio and what is necessary for the SCons-world.