Reason for change
At C&A's request: We have a new GR, GR v15r40, which contains the space craft alignment code. We also have two new sets of job options. One set is to enable the space craft alignment. The other is to turn on the TKR alignment again which was accidentally turned off on July 16 (in L1Proc 1.61). ***ADDED May 13, 2010 by anders: The TKR alignment was turned off in L1Proc 1.62, see this***
Note that while C&A has accepted the S/C alignment, there is no explicit SSC JIRA for them. I copied over the set with flavor "align" to "vanilla" which is the default flavor in L1. We need to start tracking them like we track all other calibrations. In particular to make sure any updates (if there are any) get propagated into L1 ("vanilla").
There are also general updates to the monitoring.
Test Procedure
We have processed data runs in the DEV pipeline with this version of L1Proc. C&A has looked at the difference between a reprocessed run (with S/C and TKR alignment) and the original processing (no S/C or TKR alignment). Their results are here. A direct comparison between the Multiple scattering angle for reprocessed and original processing can be found in the attachments to this JIRA: GDQMQ-252.
Rollback procedure
We can switch back to the previous version of L1Proc in case of problems.
CCB Jira
Details
L1Pipeline v1r64:
- Run the SVAC ntuple on the chunk level.
GlastRelease v15r40:
- The main change with this GR is the space craft alignment code. In addition, we have added two sets of job options: one to enable the space craft alignment and another to enable the TKR alignment that was accidentally turned off on July 16. These four lines of JO will be added to Gleam in a subsequent GR. For the moment they are appended to the usual recon JO by the L1 pipeline. (Sorry for the funky format below, but Confluence insists on interpreting the third JO line as a macro and this was the only way I found around it)
FT1Alg.CorrectForAberration = true;
FT1Alg.AlignmentFlavor = "vanilla";
Calibration.Members = {"TkrCalibAlg/TkrReconCalib"};
TkrReconCalib.calibFlavor = "vanilla";
- Systemtestsfor GR v15r40with respect to the previous production release, GR v15r39. There are no changes in the system tests.
- RM diffwith respect to GR v15r39.
dataMonitoring/FastMonCfg: v1r6p0
- All histograms at the tower level needs a _Tower added as a suffix. Relevant jira(s):GDQMQ-258
- Add spacecraft_orbit_mode and spacecraft_orbit_inSAA. Relevant jira(s): GDQMQ-195
- Remove cut at 0 for Cal plots CalX_Total_NHit_TH1 and CalX_NHit_TH1 in config.xml. Relevant jira(s): GDQMQ-230
- Remove Tick20MHzDeviation_SuperZoom_TH1 and tune Tick20MHzDeviation_Zoom_TH1 x-axis to (-300, -100) in config.xml configLCI.xml MergeHistos_FastMon.txt. Relevant jira(s): GDQMQ-246
- Change histogram name AcdHitsCounter_CableChannel_TH2 to AcdHitMap_GafeGarc_TH2, and swap axis to match the Digi definition in config.xml configLCI.xml MergeHistos_FastMon.txt. Relevant jira(s): GDQMQ-244
dataMonitoring/DigiReconCalMeritCfg: v1r2p27
- Trend the rate of TEM bugs using the FastMon_error_summary bit 23, type is rate in (monconfig_fastmon_trending.xml). Changes to address GDQMQ-254
- Corrected bug in filter for some of the compressed size rates (monconfig_digi_trending.xml).
- Add FastMon trending quantities FastMon_spacecraft_orbit_mode and FastMon_spacecraft_orbit_inSAA as ValueChange. in monconfig_fastmon_trending.xml
- Changes to address GDQMQ-195
- Check that signal > 0 in ACD PMT that vetoed before computing Log10() to fill histogram. This only affects recon histos job (monconfig_recon_histos.xml).
- Remove cut at 0 for Cal plots CalX_Total_NHit_TH1 and CalX_NHit_TH1_Tower[16]. Changes to address GDQMQ-230
- Remove Tick20MHzDeviation_SuperZoom_TH1 from merge histo digi list. Changes to address GDQMQ-246
- Remove Tick20MHzDeviation_SuperZoom_TH1 and tune Tick20MHzDeviation_Zoom_TH1 x-axis to (-300, -100). Changes to address GDQMQ-246
- Add histogram to hold pedestals from the database.
dataMonitoring/Common: v4r9p0
- Add function LookupErrorCode to Lookup the symbolic name of an error code in FastMon. Relevant jira(s): GDQMQ-257
- New variable in pGloblals to tell FastMon to dump event with an error to file : FASTMON_DUMP_ERRORS_TO_FILE is set to false, and is used only for debugging. Relevant jira(s): GDQMQ-253
- New attribute "seconds_elapsed" in the FastMon parsed by the pErrorLogger module.
- New algorithm to normalize the number of errors to the run duration added to pErrorLogger.
- Minor bug fix: the algorithm alg__values was crashing if for some reason it was not run (for instance if the plot was missing). Which should never happen, but... good to be prepared.
- Another bug fix: exception handling correctly implemented in alg__empty_bins.
- Added a new class (pErrorLogger) to apply alarms on the xml output files from the fastMon error handler and the verify module. The interface is identical to the one of the alarm handler, with the only difference that the new class operates on xml files, rather than root files (and the algorithms are obviously different). Not running into the pipeline, yet.
- Implemented the pedestal deviation in the CAL fitting tool, based on the new file from David (with the histogram of the pedestal references).
- Number of degrees of freedom and reduced chisuare added to the details dict for all the algorithms involving fitting.
- Left fitting range changed from 1.5 to 2.5 sigma for LEX8 and HEX8 CAL pedestals in pCalPedsAnalyzer.py.
- Some minor refactoring in the p*Analyzer classes to allow the derived classes to add their own histograms to the ones specified in the base class.
- Added histograms of the mean and rms values wrt the truncated average method in pCalPedsAnalyzer and pAcdPedsAnalyzer.
- Added placeholders for the pedestal deviations in pCalPedsAnalyzer and pAcdPedsAnalyzer.
- New alarm algorithm added: alg__low_high_ratio, measuring the ratio--in terms of number of entries--below and above a given pivot point. Relevant jira(s): GDQMQ-26
- All the alarms parameters with a default value (i.e. those which may not be specified in the xml file) added to the parameters dict of the algorithm so that they make it to the output xml file.
- Methods to adjust the x range in pAlarmBaseAlgorithm modified to use the getParameter() method for consistency (this way the min and max parameters make it to the output xml file).
- All algorithms modified to accept both TH1F and TH1D (and/or TH2F and TH2D where necessary).
- Test functions for all the algorithms modified as to print out the actual parameters dict, as it is shipped out to the xml file. * Relevant jira(s): GDM-133
- Output status promoted to warning (used to be undefined) in the edge finders if the edege(s) is (are) not found. The idea is that a minimu suitable number of entries is required before running an alarm on a plot; if the condition is not met the status is UNDEFINED---no need to bother. This change is intended to distinguish the cases in which the algorithm is supposed to have enough stastistics (and hence to find the edge) but it doesn't.
- Bug fix in pBaseAnalyzer: undefined variable when printing the fit parameters on the screen, was affecting the debug mode only.
- Bug fix in pBaseAnalyzer: the errors bars were not set correctly, there was an offset of -1.
- Default values for the ACDanalyzer quantites (for the unphysical empty histograms) set to 0.0 rather than -1.0
dataMonitoring/FastMon: v4r5p0
- Always do a CopyTree even when there is no cut, it seems more reliable pCustomPlotter.py
- Sort list of plots before creating them and add some logging to know which plot is cbeing done : pFastMonTreeProcessor.py
- Update getExpandedExpression() to add only _ and not a full suffix pXmlOutputList.py
- Needs FastMonCfg v1r6p0. Relevant jira(s): GDQMQ-258
- Update error handling using a new function provided by Ric to automatically log the error type for ALL iterators.
- TEM_BUG are still tagged as such in the ERR contribution but in the TKR contribution iterator UnphysStrip, BadStripOrder, UnPhysTOT and TooManyHits are not anymore counted as TEM_BUG, the bug should be catched earlier.
- The function LookupErrorCode(obj, code) is defined in Common/python/pGlobals.py v4r9p0. This means that this version of FastMon (>v4r4p0) REQUIRES Common >= v4r9p0. Relevant jira(s): GDQMQ-257
- error_summary is filled in __postEvent : pDataProcessor. Relevant jira(s): GDQMQ-25
- When FASTMON_DUMP_ERRORS_TO_FILE is set to true, the events with an error are dumped to a file calling the new method __dumpEventToFile. For debugging purpose only : pDataProcessor. Relevant jira(s): GDQMQ-253
- Add spacecraft_orbit_mode and spacecraft_orbit_inSAA, read them from magic7, pass them to the updated pSCPosition constructor, add get methods, and fill quantities in the pGeomagProcessor in pM7Parser.py pSCPosition.py pGeomagProcessor.py. Relevant jira(s): GDQMQ-195
- Add the seconds_elapsed to the error summary: pXmlErrorMerger. Relevant jira(s): GDQMQ-251
- Add the seconds_elapsed to the error summary: pErrorHandler.py
- Give number of seconds elapsed to the ErrorHandler: pDataProcessor.py
- Add ERR_TrgParityError to the handleError(), to synced with the last LDF developments, pLATcomponentIterator.py
- Bug fix in gtrcPhaseError, pERRcontributionIteratorBase.py. Relevant jira(s): GDQMQ-251
- Update old files to v3r5p6 after the tags were lost. Hopefully, everything is OK now. I hate cvs even more than before.
- Bug fix, event number was not correctly reported in pErrorHandler. Relevant jira(s): GDQMQ-197
- Change histogram name AcdHitsCounter_CableChannel_TH2 to AcdHitMap_GafeGarc_TH2, and swap axis to match the Digi definition, in pCustomPlotter
- Log the LDF version in pDataProcessor. Relevant jira(s): GDQMQ-243 and GDQMQ-244
svac/Monitor: v1r2p27
- Added call to function facilities::commonUtilities::setupEnvironment() in class runStrip_t.cxx in order to access MySQL with GRv15r40.
- Add object FastMon_error_summary to be able to trend the number of TEM bugs. Changes to address GDQMQ-254
- Add FastMon trending quantities FastMon_spacecraft_orbit_mode and FastMon_spacecraft_orbit_inSAA as ValueChange. in monconfig_fastmon_trending.xml
- Needs DigiReconCalMeritCfg v1r2p23
- Changes to address GDQMQ-195
- Bug corrected in definition of ValChange. This bug prevented using proxies with ValChange values.
dataMonitoring/AlarmsCfg: v4r7p5
- Added exception in FastMon eor for spikes and holes on tower 0, gtfe 13, layer 14.
- Upper warning limit on the recon AcdPhaMipsCorrectedAngle moved from 1.2 to 1.3 (required by the change in the fit subrange made in v4r7p
- A few parameter changed in recon eor configuration file to make it more roboust in case of low statistics.
- Limits on CalX_NHit_TH1_Tower_* (x_rms) in digi eor changed a little bit.
- Condition on the minimum number of entries added for the alarm on digi eor GPSInLock_TH1.
- Limits loosened a tiny bit on CalX_NHit_TH1_Tower_* (x_average) for FastMon eor, for consistency with the corresponding digi eor plot.
- Limits loosened on some of the calhist quantities, as the details of the algorithms are still under development.
calibTkrUtil v2r7
- Add error values to occupancy related parameters for TKR monitor to address request from data monitoring group. improve calculation of errors for zero entry cases.
Complete set of tags for L1Proc 1.64
Code Versions
GlastRelease (sim/recon) v15r40*
ScienceTools (Level 2) : v9r6p2
Science Ops (task defs, scripts):
Level 1 pipeline code and applications running in L1:
svac/L1Pipeline: v1r64*
calibTkrUtil v2r7*
calibGenTKR v4r5
dataMonitoring/AlarmsCfg: v4r7p5*
dataMonitoring/FastMonCfg: v1r6p0*
dataMonitoring/DigiReconCalMeritCfg: v1r2p27*
dataMonitoring/Common: v4r9p0*
dataMonitoring/FastMon: v4r5p0*
datMonitoring/IGRF: v1r0p1
svac/Monitor: v1r2p27*
svac/EngineeringModelRoot: v4r3
svac/TestReport: v6r16
users/richard/pipelineDatasets: v0r6
ft2Util: v1r2p23
evtClassDefs v0r6
GPLtools: v1r11