SCons discussion July 2008

July 9, 2008

Attendees: Joanne Bogart, Toby Burnett, Emmanuel Cephas, Richard Dubois, Tom Glanzman, Navid Golpayegani, Heather Kelly, Tracy Usher, Eric Winter

ScienceTools Status

Twelve unit tests (out of about 70) in ST continue to fail. Many are due to issues with hoops and apes, and once those are sorted out, most will be fixed. Some of the failing tests belong to Toby. Navid and Toby should get together to discuss.

What is left is to recreate the installer scripts - but that is not currently a high priority.

GlastRelease

Navid has SConsified AcdDigi and AcdUtil to the point where he can run their unit tests. This of course involved creating SConscript files for all the package dependencies as well. This is done on u30, and Navid will commit the changes to CVS shortly. (In fact that may already be done) The next step would hopefully entail involvement from the developers, otherwise Navid will step in and do it himself. Certainly there is interest from some developers (Joanne and Heather come to mind) to help out. It was suggested that to apply some additional pressure, we could use the new RM to highlight what packages are not ready for primetime in SCons. Unfortunately, Navid points out that currently the new RM reports packages as passed when they do not contain SConscript files. Also missing unit tests are not flagged as they are now with the current RM. Navid will be addressing these issues soon.

Next Navid hopes to test the SConscript files for GR on Windows. Tracy is also interested in learning how to grab what Navid has done so far, so he can try it out. Toby hopes Tracy will show him what he learns.

Here is the list of currently modified packages in CVS:

xmlUtil
lsfData
idents
gui
geometry
enums
detModel
commonRootData
calibUtil
calibRootData
TkrUtil
LdfEvent
GuiSvc
GlastSvc
Event
CalibSvc
CalibData
CHS/eventFile
AcdUtil
AcdRecon
AcdDigi

JO Files

Currently our JO files can refer to each other through the use of CMT defined environment variables. We need to move to our new mechanism to define env vars at runtime via the facilities package, which is backwards compatible with CMT. This does mean that the <package>ROOT env variables will no longer be created once we complete the move to SCons. Instead, all the environment variables will be based off one variable, as is done in ST called INST_DIR. Eric W. noted that this will greatly simplify the existing ST setup scripts which manually set the environment variables.

Navid presented his proposal that the JO files be handled as we currently handle the XML files in ST. For the XML files, we have a top-level directory called xml, with subdirectories for each package. Within each package subdirectory, the set of XML files for that package are copied over. This is a one-level directory structure - with no subdirectories under the package directories. So we would have a structure something like this:

Continuing with the ST XML file example, some of the xml files were not necessarily named or located using any particular convention. There were conflicting file names within the same package, for example. Similar issues plague the current set of JO files in GR. The SConscript files currently list all the XML files for each package, they are then gathered up during the build and copied into the top-level xml/package sub directory. There is then an environment variable defined such as $packageXMLPath for each package. Tracy wondered aloud why we need to move the JO files to a top-level directory when we are still defining an env variable for each package. Navid reminded him that we wanted to use this move to SCons as an opportunity to clean things up and also this helps address the issue of lengthy PATHs which were causing trouble in some environments.

Tom asked about how we would handle having a locally modified package built against a central GR build (Overriding packages). Navid replied that we could checkout a package into a local directory, and then when we call SCons to do the build, we will pass an additional option to specify that packageX is located in a different directory. Heather asked about handling lower level packages, such as facilities, which everyone else depends upon. SCons would not go off and attempt to rebuild all the packages that depend on facilities. If we desire to rebuild those packages, we could use the SCons dependency tree (like CMT show uses) to determine what packages are involved and create local versions to rebuild.

Navid outlined a possible migration path to minimize pain: We could introdude the new environment variables and continue to define the CMT environments as we do now. Then update the JO files to use the new variables, followed by eliminating the CMT environment variables.

External Libraries

Currently we are not consistently applying any convention to our external libraries. Navid would like to change that with this move to SCons. For every external we will have something like:

externalPackageName
|
-----version
           |
           ----compiler version
                          |
                          ------include
                          |
                          ------lib

GLAST_EXT would be a parameter passed to SCons for the build. For end-users and HEASARC, we would continue to define GLAST_EXT.

Speaking of HEASARC, Navid mentioned he hopes to create a SCons builder which will generate the hmake Makefiles.

Next Step

Richard suggested that we should scope out the extent of the problem for JO files. Tom suggested that we could migrate to having a jobOptions directory in all packages, as part of our effort to move to a convention.

Windows and SCons and other issues

We have moved to a more recent version of SCons. The SCons team is preparing for a version 1.0 release and is in bug-fixing mode. Navid reports that he can create project files and open them up in VC++ and debug as well as develop and rebuild. This is different than the experience Tracy and Toby reported over a year ago with an older version of SCons. There was still a question as to whether solution files for a package would be aware of dependent project files. This should be tested again.

Rootcint has been handled for ST, and Navid reports he has applied it to commonRootData in GR and it seems to work.

Swig was done for ST. Toby asked if this was Jim's Swig or his Swig - we suspect Jim's Swig. Toby and Navid should discuss what the difference is.

We will discuss timing of introducing the new tagging convention after L&EO. Currently we intend to that around mid-August.

Child pages