Overview
This is a description of the backend of the system tests. They consist of a perl module, several perl scripts and some root macros. The bulk of the original development was by Karl Young.
The purpose of the GLAST SAS system tests is to provide and end to end test of the offline software. They are typically run for each tagged release of the GlastRelease and EngineeringModel packages.
The components are outlined in the following diagram.
The main executive script (runSystests.pl) is run manually at the moment. It runs the suite of shell scripts that it finds in the EMtests and GRtests subdirectories, tracks results and updates database tables. Each of the shell scripts that it runs consists of at least two parts, it first runs either a Gleam.exe or LatIntegration job then runs a RootAnalysis macro which reads the output files produced in the Gleam step and fills a collection of histograms. Once the histograms have been created, the executive script sets the standard and threshold for each histogram, that is, the version that this release is to be compared against, and the statistical threshold that defines whether the test has passed or failed. Finally the executive script runs a root macro which calculates several quantities from each histogram (number or entries, rms, KS probability that the histogram differs from its standard etc) and updates the database.
The system tests code can be found in u17/systests/src. The EMtests and GRtests directories contain one directory for each test type used for the EngineeringModel and GlastRelease packages respectively. The exec directory contains the perl scripts that run the tests.
Running the system tests
The system tests are configured using a configuration file. An anotated example can be found in /nfs/farm/g/glast/u17/systests/src/exec/Config_example.txt.
Running all parts of the system tests
To run the system tests from scratch, i.e. runing Gleam to create the full root trees, RootAnalysis to create the histograms and finally a step to set the standard version, threshold and calculate the metadata, do (from the exec directory):
./runSystest.pl <configfile>
Where <configfile> is the name of your configuration file. If you do not supply a config file name, the code will look for a file called Config.txt by default. It is important that only one set of system tests is running at any given time