Page History

Goal

Provide RM builds of GRBAnalysis for distribution.  GRBAnalysis depends somewhat loosely on ScienceTools: the BackgroundEstimator subpackage links to a few ST libraries, and other elements within GRBAnalysis make use of ST binaries.

Method

1. Make a ScienceTools external library.  That is, for current ST tag and others in the future as needed, install the ST build (libraries, binaries, includes, python, and any other public files as needed) in the SLAC externals area, rearranging the file hierarchy somewhat to be consistent with norms in the Externals area. Add entries to allExternals.scons for this new external (known as STExt)
2. Re-organize GRBAnalysis into standard form for a container (aka checkout package).  It will depend on STExt as well as on all the externals which ScienceTools itself depends upon.

Status

As of May 9th..

STExt

I've written a script and committed to CVS (grits-tools/python/InstallST.py) to be used to install the STExt external at SLAC for all supported Linux platforms.  Here is the help:

jrb@noric10 $ python InstallST.py --help
Usage: InstallST.py version-to-install [options]  

Options:
  -h, --help            show this help message and exit
  -a ARCH, --arch=ARCH, --architecture=ARCH
                        32 or 64  [default:value for log-in machine ]
  -o OPSYS, --os=OPSYS  rhel4 or rhel5 [default:value for log-in machine ]
  --opt, --optimized    use optimized build of ScienceTools [default] 
  --debug, --noopt      use debug build of ScienceTools
  --parent=PARENT       parent directory [default: $GLAST_EXT/STExt]
  --fake                print proposed actions but don't execute
  --real                really do it

However it doesn't yet behave entirely as advertised.  In the meantime, I've made a rhel4 32-bit STExt external for the 09-23-01 tag and have installed it in the central GLAST_EXT area. See /afs/slac/g/glast/ground/GLAST_EXT/redhat4-i686-32bit-gcc34/STExt/09.23.01 (Externals aficionados may notice that the extra compiler level - gcc34 in this case - has been omitted from the directory structure.  This was inadvertant at first, but I left it alone since there seemed to be no harm in it. The entry in allExternals has been written consistent with this organization.)

InstallST.py is now working (as of May 10).  I used it to install the 64-bit rhel4 and rhel5 builds of 09-23-01.  Could not install rhel5 32-bit because the GLAST_EXT disk for that platform was too full already.

STExt has all the installed libraries, binaries (executables; not wrapper scripts), includes, python files, data files, and system pfiles from ScienceTools-09-23-01.  Is this the right set of things?

SCons Description of STExt

Since GRBAnalysis only requires a small subset of the ScienceTools libraries (facilities, tip, astro. *Respose, irfInterface, irfLoader and rootIrfLoader) I defined a library group in allExternals.scons consisting of just these libraries called STforGRBLibs.  The SCons configure step fails, I suspect because of the mixture of shared and static libraries. It is almost certainly an artifact of the particular way the configure check is done, and not because anything is actually wrong with the libraries. For the time being I've added a new type of key in allExternals.scons allowing the definition of a library group without doing the configure check.

GRBAnalysis-scons

GRBAnalysis needs to be reorganized into a standard container.  Much of the work has been done, but not yet committed to CVS.  The BackgroundEstimator shareable builds without error against STExt. 

Still To Do

Create a new container (aka checkout package), GRBAnalysis-scons, consisting of the GRBAnalysis code plus and packages from ST needed at build time.  Do not incorporate packages from ST if they only provide binaries invoked GRBAnalysis.  Assume instead that anyone using GRBAnalysis will have done setup for some ST build and will make use of binaries belonging to that build.

Status

As of June 10th..

GRBAnalysis-scons exists in CVS and contains nearly all source files from the original GRBAnalysis module.  

As of May 27th..

See /nfs/farm/g/glast/u54/jrb/GRBAnalysis-scons.   GRBAnalysis has been augmented with ST packages facilities, astro, tip, embed_python, st_facilities, st_stream and irfs/* as well as infrastructure required for SCons containers.  I've made some minor changes to astro and facilities (committed and tagged) so they will build static libraries in the context of GRBAnalysis (but not ST or GR).  I made other small changes to handoff_response and irfLoader (likewise committed and tagged) to reduce the GRBAnalysis dependence on ST packages to the minimum.

Still to do

• Add new target type so that arbitrary scripts may be installed in bin directory. (This isn't specific to GRBAnalysis; it will be available to all SCons checkout packages.)
• Add LATBA subpackage once proper organization is determined
• Tag all GRBAnalysis subpackages in the new module (e.g. GRBAnalysis-scons/BackgroundEstimator, GRBAnalysis-scons/LLE, etc.)
• Create packageList.txt file for GRBAnalysis-scons enumerating all packages and their tags to be included in a build
• Add test program(s) Optional, but recommended at least for BackgroundEstimator
• Modify cvs tagger program to treat GRBAnalsys container-wide tags as it does ScienceTools and other recognized containers
• Create container-wide tags using tagCollector script

  Wiki Markup
  {bgcolor: #c0f0c0} Made GRBAnalysis-HEAD-1-3{bgcolor}

• Add SCons RM database entries in order to automate builds of various kinds (LATEST, HEAD, release tags) on platforms of interest
• Finalize organization in CVS and naming.  Propose creating new top-level CVS module GRBAnalysis-scons which will contain everything currently in GRBAnalysis, but with source file names changed to be consistent with standard conventions (e.g. .h rather than .hh; .cxx rather than .cc). Also add sym links in the repository for ST packages to be included.

  Wiki Markup
  {bgcolor: #c0f0c0}As of June 2nd GRBAnalysis-scons has been created in CVS but so far contains only BackgroundEstimator, BayesianBlocks and containerSettings, plus sym links to some ST packages and to SConsFiles. This is [sufficient to build the BackgroundEstimator library|^GRBAnalysis-scons.png].{bgcolor}

  Wiki Markup
  {bgcolor: #c0f0c0} As of June 10th all subpackages have been added.{bgcolor}

• Add SConscripts for GBMtoolkit, GtGRB, etc.(all  subdirectories of original GRBAnalysis  which don't yet have an SConscript)
• Determine exactly what needs to be built and installed or just installed; update SConscripts accordingly. See section "Input Needed" below for more detail
• Test that all build products get properly built and installed and can run when user has set up environment as intended.  That is,
  • set environment variable INST_DIR to root directory of ScienceTools build
  • source $INST_DIR/bin/redhat../_setup.sh (or .csh)
  • set environment variable INST_DIR to root directory of GRBAnalysis-scons
  • source $INST_DIR/bin/redhat.../_setup.sh
• Create and install STExt for all other supported platforms for 09-23-01. 

• Wiki Markup
  \[Would be nice\] -Fix InstallST.py and- Enlarge scope to include Mac as well as all Linux platforms so the process will be more automatic the next time a new STExt release is needed. Maybe write a separate, but relatively simple, script to handle Windows only.

• Wiki Markup
  \[Would be nice\] Find a way to re-instate library configure checks for  STforGRBLibs

• Finalize file organization, naming, etc. within GRBAnalysis (which will probably have to become GRBAnalysis-scons).
• Add SConscripts for subdirectories of GRBAnalysis which don't yet have them
• Determine exactly what build products from GRBAnalysis need to be installed; update SConscripts accordingly.
• Test that all build products get properly built and can run.  Check that STExt contains everything needed by GRBAnalysis.

Input Needed

For each subdirectory of GRBAnalysis I need to know what needs to be built (also how, in some cases) and what needs to be installed in order to organize the files according to standard conventions and write a suitable SConscript.

...

GBMtoolkit (owner: Giacomo)  Would like to make subdirectories src (for .C) and python (for .py).  .so file does not belong in CVS. What command or commands are used to create the .so?  This will need to be in the SConscript. Typically an .so is specified as some type of library target and is installed under the top-level lib directory.  That directory will be included in LD_LIBRARY_PATH for users who run the _setup file. 

Should all the python files be installed?  If so, maybe should rename ftools.py to something less likely to cause confusion or conflicts. Once all this is straightened out, should be straightforward to make an SConscript.

...

LLE (owner: Frederic) Usual comments:  move all files except SConscript, .cvsignore out of top directory.  Determine what needs to be built; what needs to be installed. I haven't tried it yet, but the The SConscript and file LLELib.py in there now look right for building build the LLE library. Will have to add to SConscript if other files belonging to the package need to be installed.

...

There are several files throughout GRBAnalysis with no comments whatsoever.  Usually, but not always, the name of the file is informative, but this really isn't enough for anything you expect other people to use.  Please add a sentence or two at the top, explaining briefly what the file is for, inputs and outputs and other similar information.

Old Stuff

Most likely obsolete. I'm preserving this text in case we decide there is some value in an STExt, either for GRBAnalysis or in some other circumstances.

{bgcolor:#a0a0a0}

h4. Method

# Make a ScienceTools external library.  That is, for current  ST tag and others in the future as needed, install the ST build  (libraries, binaries, includes, python, and any other public files as  needed) in the SLAC externals area, rearranging the file hierarchy  somewhat to be consistent with norms in the Externals area. Add entries  to allExternals.scons for this new external (known as STExt)
# Re-organize GRBAnalysis into standard form for a container  (aka checkout package).  It will depend on STExt as well as on all the  externals which ScienceTools itself depends upon.

h4. STExt

I've written a script and committed to CVS  (grits-tools/python/InstallST.py) to be used to install the STExt  external at SLAC for all supported Linux platforms.  Here is the help:

jrb@noric10 $ python InstallST.py \--help
Usage: InstallST.py version-to-install \[options\]  

Options:
  \-h, \--help            show this help message and exit
  \-a ARCH, \--arch=ARCH, \--architecture=ARCH
                        32 or 64  \[default:value for log-in machine\]
  \-o OPSYS, \--os=OPSYS  rhel4 or rhel5 \[default:value for log-in machine\]
  \--opt, \--optimized    use optimized build of ScienceTools \[default\] 
\--debug, \--noopt      use debug build of ScienceTools
  \--parent=PARENT       parent directory \[\]
  \--fake                print proposed actions but don't execute
  \--real                really do it
\\
I've made a rhel4 32-bit STExt external for the 09-23-01  tag and have installed it in the central GLAST_EXT area. See  /afs/slac/g/glast/ground/GLAST_EXT/redhat4-i686-32bit-gcc34/STExt/09.23.01  (Externals aficionados may notice that the extra compiler level - gcc34  in this case - has been omitted from the directory structure.  This was  inadvertent at first, but I left it alone since there seemed to be no  harm in it. The entry in allExternals has been written consistent with  this organization.)

InstallST.py is now working (as of May 10).  I used it to install the  64-bit rhel4 and rhel5 builds of 09-23-01.  Could not install rhel5  32-bit because the GLAST_EXT disk for that platform was too full  already.

STExt has all the installed libraries, binaries (executables; not  wrapper scripts), includes, python files, data files, and system pfiles  from ScienceTools-09-23-01.  Is this the right set of things?

h4. SCons Description of STExt

Since GRBAnalysis only requires a small subset of the ScienceTools  libraries (facilities, tip, astro. \*Respose, irfInterface, irfLoader and  rootIrfLoader) I defined a library group in allExternals.scons  consisting of just these libraries called STforGRBLibs.  The SCons  configure step fails, I suspect because of the mixture of shared and  static libraries. It is almost certainly an artifact of the particular  way the configure check is done, and not because anything is actually  wrong with the libraries. For the time being I've added a new type of  key in allExternals.scons allowing the definition of a library group  without doing the configure check.{bgcolor}