Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

The code for the system tests currently resides in /nfs/farm/g/glast/u17/systests/src. (TBD: commit the current working version of the code to CVS!) 

The main executive script is runSysTests.pl in the the exec subdirectory, and is supported by utility routines in SysTests.pm. It runs the suite of shell scripts that it finds in the EMtests, GRtests and BTtests subdirectories of src/, tracks the results and updates Oracle database tables. Each of the shell scripts that it runs consists of at least two parts: it first runs either a Gleam or LatIntegration job, and  then runs a RootAnalysis macro, which reads the output files produced in the Gleam step and fills a collection of histograms. Once the histograms have been created, the executive script sets the standard and threshold for each histogram, that is, the version that this release is to be compared against, and the statistical threshold that defines whether the test has passed or failed. Finally the executive script runs a root macro which calculates several quantities from each histogram (number or entries, rms, KS probability that the histogram differs from its standard etc) and updates the database.

The EMtests, GRtests and BTtests directories contain one directory for each test type used for the EngineeringModel, GlastRelease and BeamtestRelease packages respectively. The exec directory contains the perl scripts that run the tests. The output of the tests (root files and run log and summary) ends up in subdirectories of GlastRelease/, EngMod/ or BeamtestRelease/.

Running the system tests

The system tests are configured using a configuration file. An anotated example can be found in /nfs/farm/g/glast/u17/systests/src/exec/Config_example.txt.

Running all parts of the system tests

...

Go to the directory which will contain the output. For example, to run the test for v11r3 of GlastRelease, go to:

    /nfs/farm/g/glast/systests/GlastRelease/v11r3.

In this directory, you will find the file Config.txt, which was put there by the Release Manager. If it isn't there, copy one from a previously run suite of systests. Modify this file as desired. For example, you might want to change the reference release, or add the subset command. The reference chosen must have previously run successfully.

...

Wiki Markup
    _/nfs/farm/g/glast/u17/systests/src/exec/runSysTests.pl \[_configfile_\]_

where configfile is the name of your configuration file. If configfile is not supplied, Config.txt will be used.

...

The output for this run will appear in the directories:

    /nfs/farm/g/glast/u17/systests/GlastRelease/v11r3/testname/linux

where testname is AllGamma, VerticalMuon1GeV, etc. 

In slightly more detail, runSysTests.pl launches a batch job, which runs a series of shell scripts, say GRtests/AllGamma/systest_AllGamma, as an example. This job runs Gleam using the jobOptions.txt file in the same directory, then runs the RUN_linux root macro in that directory to generate the plots. Throughout this process, the Oracle database is updated as the tests progress.

It is important that only one set of system tests for each package is running at any given time. You can run a set of system tests for EngModel and GlastRelease simultaneously, but you cannot run two sets for different GlastRelease versions, because the tests will try to store the output files in the same working directory. After you start the system tests it is usually a good idea to look at the file testerr (in the same directory that you ran from). We fairly often get a sporadic error when connecting to the Oracle database. If this happens, you should just kill the jobs with bkill 0 and start them again.

...

If the full root trees already exist and are stored in the systests db and you wish to regenerate the histograms you can do:
.

Wiki Markup
  _/nfs/farm/glast/g/u17/systests/src/exec/runRootOnly.pl
<configfile>
 \[_configfile_\]_

This will access the database to find the locations of the files containing the root trees and will regenerate the histograms. This is very convenient if the RootAnalysis macros have been altered.

...