...
This is a description of the backend back end of the system tests. They consist of a perl module, several perl scripts and some root macros. The bulk of the original development was by Karl Young. You can find a copy of the documentation he wrote attached to this page (you'll notice that these pages bear some similarities to his work!)
...
The code for the system tests currently resides in /nfs/farm/g/glast/u17/systests/src. (TBD: commit the current working version of the code to CVS!)
NB: if you have logged in as glastsys, the envvar $SYSTESTS is set to /nfs/farm/g/glast/u16/systests.
The main executive script is runSysTests.pl in the the exec subdirectory, and is supported by utility routines in SysTests.pm. It runs the suite of shell scripts that it finds in the EMtests, GRtests and BTtests subdirectories of src/, tracks the results and updates Oracle database tables. Each of the shell scripts that it runs consists of at least two parts: it first runs either a Gleam or LatIntegration job, and then runs a RootAnalysis macro, which reads the output files produced in the Gleam step and fills a collection of histograms. Once the histograms have been created, the executive script sets the standard and threshold for each histogram, that is, the version that this release is to be compared against, and the statistical threshold that defines whether the test has passed or failed. Finally the executive script runs a root macro which calculates several quantities from each histogram (number or entries, rms, KS probability that the histogram differs from its standard etc) and updates the database.
...
The system tests are configured using a configuration file. An anotated annotated example can be found in /nfs/farm/g/glast/u17/systests/src/exec/Config_example.txt.
...
To run the system tests from scratch, i.e. runing running Gleam to create the full root trees, RootAnalysis to create the histograms and finally a step to set the standard version, threshold and calculate the metadata:
Go to the directory which will contain the output. For example, to run the test for v11r3 of GlastRelease, go to:
/nfs/farm/g/glast/systests/ > cd $SYSTESTS/GlastRelease/v11r3.
In this directory, you will find (or not) the file Config.txt, which was put there (or not) by the Release Manager. If it isn't there, copy one from a previously run suite of systests. Modify this file as desired. For example, you might want to change the reference release, or add the subset command. The reference chosen must have previously run successfully.
To start the tests, do:
| Wiki Markup |
|---|
_/nfs/farm/g/glast/u17/systests/_> $SYSTESTS/src/exec/runSysTests.pl \[_ configfile _\]_ |
...
The output for this run will appear in the directories:
/nfs/farm/g/glast/u17/systests/ > $SYSTESTS/GlastRelease/v11r3/testname/linux
...
If the full root trees already exist and are stored in the systests db and you wish to regenerate the histograms you can do:
| Wiki Markup |
|---|
_/nfs/farm/glast/g/u17/systests/ _> $SYSTESTS/src/exec/runRootOnly.pl \[_ configfile _\]_ |
This will access the database to find the locations of the files containing the root trees and will regenerate the histograms. This is very convenient if the RootAnalysis macros have been altered.
Making the histograms appear (no longer needed)
At the moment, the step to calculate the metadata and insert it into the database is not automatically run (due to a mismatch in oracle versions needed for the ROOT/perl step). The histograms cannot be viewed until the metadata is in the database. To run this manually, you need to login as glastsys,
cd to u17/systests/src/exec directory
copy the Config.txt that you need to the current directory
i.e. cp ../../EngModel/v5r0703p5/Config.txt .
and run
Re-running the comparisons
Once the gleam and root jobs have run, and the output files and histograms have been created, the comparison can be re-run quickly. In fact new comparisons between releases, or other special runs can be done in the same way:
Generate the appropriate Config.txt file in the directory /src/exec, and edit as desired:
> cd $SYSTESTS/src/exec
> cp $SYSTESTS/GlastRelease/v12r4/Config.txt Config_GRv12r4.txt
> emacs Config_GRv12r4.txt &
> ./ insert_metadata.pl Config_GRv12r4.txt