Validating GR for RHEL4

Michael Kuss is kindly leading the charge to check that our simulation results when running RHEL4 builds of GlastRelease are at statistically equivalent to our RHEL3 builds.

Simulations Run

GR v17r34p0rhel4B:  http://glast-ground.slac.stanford.edu/SystemTests/?releaseVersionId=11881  OutputLevel was set to Debug.
Use v17r34p0 as a reference which is the RHEL3 version of the systests.  Please note v17r34p0B is another RHEL3 run and the results are seemingly identical..though OutputLevel was set to Debug.  The resulting log file may be of interest.

Of particular interest has been the BackGndMixDC2 test.

Full ROOT files are available on u17/systests/GlastRelease where one uses the name of the systest to drill down into the directory structure.

System test report with more plots is here

Calling All GR Package Owners

It is requested that all GR Package owners take a hard look at the warnings associated with their packages in the RHEL4 builds.  Not sure what packages you own??  Check the list:

https://confluence.slac.stanford.edu/display/SAS/List+of+GR+Package+Owners+as+of+September+8+2009

Questions Swirl Around MCTERMZ

Looking at BackGndMixDC2, one failed plot is MCTERMZ.

However, for the 1 GeV Proton test, this distribution is almost identical for the rh9 and the rhel 4 runs.

Unsigned versus Signed Ints

There are a number of warnings in our builds concerning unsigned/signed match.  These should be looked over and fixed.

An audit of existing JO properties is also in order to check for mismatches, as these will not be caught at compile time.

One specific JO issue concerns TriggerAlg.mask:

Trigger: there is something really weird with the trigger mask:

In the jobOptions:
TriggerAlg.mask = -1;

rh9_gcc32opt logs:
TriggerAlg           INFO No trigger requirement
rhel_gcc34opt logs:
TriggerAlg           INFO Applying trigger mask: 589cba38

In TriggerAlg::TriggerAlg() we find:
    declareProperty("mask"    ,              m_mask=0xffffffff);

and later, in TriggerAlg::initialize()
log << MSG::INFO;
    if(log.isActive()) {  
        if (m_mask==0xffffffff) log.stream() << "No trigger requirement";  
        else            log.stream() << "Applying trigger mask: " <<  std::setbase(16) <<m_mask <<std::setbase(10);  
        if( m_throttle) log.stream() <<", throttled by rejecting  the value "<< m_vetobits;
    }
log << endreq;

mask is an unsigned int.  For the moment, could we rerun both systests with simply the Trigger.mask=-1; line commented, as all bits set is the default anyway? And note anyway that the code should be changed?  rh9 interprets correctly in the sense that with (signed int)-1 0xffffffff is intended.  How rhel4 comes up with 0x589cba38 beats me.

Suggested Fix:

Make TriggerAlg.m_mask a StringProperty and convert the value to an unsigned int via the tools available in facilities::Util.  This has been tried out and seems to work.

Current Status:

Unfortunately, the systest results remain the same (comparing v17r34p0 versus v17r34p0B and v17r34p0rhel4 versus v17r34p0rhel4B, even after being sure that the Trigger.mask is set to the appropriate default.

Exception in CalLikelihoodManagerTool

rhel4 also contains one event that threw an exception in CalLikelihoodManagerTool

Random Sequences Diverge

In regards to the BackGndMixDC2 run and the mix of events:

"As Richard mentioned, the two random sequences will diverge eventually. Hence, the particle mix for the 40k triggers is different.  The biggest discrepancy is in "1002 CrElectronSplash", with 6409 trigger for rh9_gcc32opt vs. 6603 for rhel4.  Consequently, the other sources are weighted stronger in rh9, in general."