Introduction
The System Tests track the performance of the Gleam application as it evolves through its code releases. It can run several configurations of Gleam (eg AllGamma, several energies of vertical gammas and muons, etc.) and compare histograms to previous standard versions.
The original version, System Tests I, used ASP pages, VBScript to talk directly to the Oracle database, and parts of JAS wrapped in a JSP Servlet to display the plots.
A newer version is currently in development that is part of a comprehensive suite of packages named GRITS that is designed to support the infrastructure needs of SAS.
Usage
Selecting a version of GlastRelease (which sets the Gleam version) will present a table of tests for that version and list the following information:
- CPU time in seconds
- memory used in MB
- link to histograms overlaid with a well-known standard
- link to the merit ntuple (ROOT file) for this test
- link to a table of the following metadata associated with each histogram:
- mean
- RMS
- number of entries
- KS test value
- link to histograms that pass the KS test (with a KS value above a threshold of 0.5)
- link to histograms that fail the KS test (with a KS value below a threshold of 0.5)
- summary count of these two conditions
Note: It is now known that the KS Test is wrong for comparing binned data (i.e. histograms). Although the KS Test is approximately right for some cases of binned data, there are other more statistically rigorous ways pf calculating the similiarity of histograms that will be used in System Tests II. You will notice that System Tests I sometimes erroneously flags hisgorams as deviating from the standard when, upon inspection, the histograms are actually satistically identical. This is a known probelm for System Tests I.
Selecting a particular test's histograms will present a pull-down menu of histogram titles available in the Root file and a button to enable overlays.
Caveats
April 20, 2003
Not all the System Tests are working, so avoid those with cpu times of zero. The test names should be self-explanatory. KS test thresholds have been set to 0.5 for now.
The Plot Histos link allows an individual histogram to be overlaid with a standard. The new statistical tests mentioned above will flag histograms that are statistically different from the standard and make it dramatically simpler to identify problems. For now it is a bit tedious to march through all the histos.
Although the histogram names should be reasonably self explanatory, the source code may also be consulted.