Blog from January, 2010

Request to deploy L1Proc 1.82
Unknown User (wbfocke) posted on Jan 15, 2010

Reason for change

To update monitoring code.

Test Procedure

We have processed runs in the DEV pipeline with this version of L1Proc.

Rollback procedure

We can easily switch back to the previous version of L1Proc.

CCB Jira

SSC-242@JIRA

Details

L1Pipeline: L1Pipeline v1r82

Complete set of tags for L1Proc 1.82
Code Versions
GlastRelease (sim/recon): GlastRelease-v15r47p12gr10*
ScienceTools (Level 2) : v9r15p5
Science Ops (task defs, scripts):
Level 1 pipeline code and applications running in L1:

svac/L1Pipeline: L1Pipeline v1r82*

calibTkrUtil v2r9p1
calibGenTKR v4r5

dataMonitoring/AlarmsCfg: AlarmsCfg-05-22-05

  • AlarmsCfg-05-22-05 27-Jan-2010 lbaldini Minor changes.
    • Condition in min entries for CondSummaryWord_TH1 (reference histogram, digi and FastMon eor) changed from 10000 to 25000.
    • A couple of not-so-obsolete exceptions on FastMon eor back in.
  • AlarmsCfg-05-22-04 25-Jan-2010 lbaldini Minor change.
    • Condition on min_entries for the alarms on CondArrCNO_CalLoOpensWindow_TH1 changed from 1000 to 3000 (for both FastMon and digi eor).
  • AlarmsCfg-05-22-03 24-Jan-2010 lbaldini Minor change.
    • Minor limit change for one of the alarms on CondArrROI_CNOOpensWindow_TH1 (low_high_ratio with pivot 31) for both FastMon and digi end of run: 
      <       <warning_limits min="5.0" max="20.0"/>
<       <error_limits   min="2.0" max="40.0"/>
---
>       <warning_limits min="5.0" max="30.0"/>
>       <error_limits   min="2.0" max="50.0"/>
    • See comment Science_Operations/General_Comments - Log - 7071 on the ops log.
  • AlarmsCfg-05-22-02 24-Jan-2010 lbaldini New parameter max_rel_err used for the alarm on towerEff in tkrTrending.
    • New parameter max_rel_err of alg__values.py used for the alarm on towerEff in tkrmon trending (set to 3%).
    • Relevant Jira(s): GDQMQ-338
  • AlarmsCfg-05-22-01 22-Jan-2010 lbaldini Limits on OutF_Normalized_AcdHit_AcdTile tuned
    • Limits on OutF_Normalized_AcdHit_AcdTile tuned. This is meant to have a systematic log of solar flares causing an increase in the hit rate on the +X side of the ACD.
  • AlarmsCfg-05-22-00 21-Jan-2010 lbaldini Obsolete alarm exceptions removed.
    • Obsolete exceptions removed.
    • Relevant Jira(s): GDQMQ-337
  • AlarmsCfg-05-21-02 19-Jan-2010 lbaldini A couple of minor limit changes.
    • Minor changes in the upper limits for the CondArrCNO_CalHiOpensWindow_TH1 quantity (in digi and FastMon eor) for the low_high_ratio (with the pivot at 31). Warning: 10000 -> 15000, error: 15000 -> 20000.
  • AlarmsCfg-05-21-01 15-Jan-2010 lbaldini New alarm added.
    • Added a new alarm on Mean_Tick20MHzDeviation.
    • Relevant Jira(s): GDQMQ-331
  • AlarmsCfg-05-21-00 12-Jan-2010 lbaldini Alarms on the 20 MHz tick deviation retuned/added.
    • Limits on digi eor Tick20MHzDeviation_TH1 changed to -200; -192; -176; -168; for the x_average algorithm;0.0; 0.0; 2.0; 3.5; for the x_rms algorithm.
    • For consistency the same alarm has been added to the FastMon, along with an additional one on DeltaEventTime_Zoom_TH1 (with the same limits used in the digi eor).
    • Relevant Jira(s): GDQMQ-331
  • AlarmsCfg-05-20-03 16-Oct-2009 lbaldini Changed limits on the uncompressed/compressed event size)
    • Limits on digi_trend OutF_Ratio_EvtSize_CompressedEvtSize changed after the enabling of the diagnostics.

dataMonitoring/FastMonCfg: FastMonCfg-02-01-01

  • FastMonCfg-02-01-01 15-Jan-2010 bregeon minor change for GDQMQ-325
    • Change gem_delta_event_time type, fix PrescaledDelta histo - GDQMQ-325
  • FastMonCfg-02-01-00 12-Jan-2010 lbaldini Cut on the error_summary changed from <64 to <256
    • Bug fix in the cuts used to prevent the events with errors from ending up in the FastMon eor plots. In the file config.xml 64 occurrences of "error_summary<64" have been changed into "error_summary<256".

dataMonitoring/DigiReconCalMeritCfg: DigiReconCalMeritCfg-01-04-06

  • DigiReconCalMeritCfg-01-04-06 18-Jan-2010 bregeon Handling earth limb correction in normalized rates
    • Added two new configuration files with the new normalization factors for the merit trending: 
      FactorsToNormRates_noEarthLimb.txt
      format identical to the one we've been using so far. 
      FactorsToNormRates_EarthLimb.txt
      new format with the four parameters for the correction for the Earth limb in the FOV.
    • Added Mean_PtSCzenith to all the relevant lines in monconfig_merit_trending.xml (to be used in the Earth limb correction for the normalized rates). Requires the new svac/Monitoring code Monitor-01-05-00.
    • Relevant Jira(s): GDQMQ-310, GDQMQ-317, GDQMQ-330
  • DigiReconCalMeritCfg-01-04-05 15-Jan-2010 lbaldini Three new quantities.
    • Added three new quantities (actually errors on quantities that were already in).
    • Relevant Jira(s): GDQMQ-327
  • DigiReconCalMeritCfg-01-04-04 15-Jan-2010 bregeon Branch tag - Add GoodEvent cuts in digi histo and remove CondSummaryWord_WithPeriodic_TH1 - GDQMQ-325
    • Remove CondSummaryWord_WithPeriodic_TH1 and add cut for error events and periodic trigger on most histograms to match FastMon - GDQMQ-325

dataMonitoring/Common: Common-06-07-00

  • Common-06-07-00 24-Jan-2010 lbaldini New max_rel_err parameter for alg__values.py implemented.
    • New parameter for the alg__values.py alarm algorithm implemented in order to disengage the alarm on any single data point whose relative error is larger than an adjustable threshold (used in the tracker trending alarms in to prevent warnings/errors from firing with low statistics).
    • Relevant Jira(s): GDQMQ-338
  • Common-06-06-01 19-Jan-2010 lbaldini Minor bug fix.
    • "exception violations" changed in "exception_violations" on line 426 of file pAlarmBaseAlgorithm.py. It's a bug that's been probably there since the dawn of times, as there's already a field "exception_violations" filled in pAlarmOutput.py and they should really point to the same thing.
  • Common-06-06-00 15-Jan-2010 lbaldini Minor change in number formatting.
    • Minor change in the routine formatting the output numbers from the alarm handler.
  • Common-06-05-04 14-Jan-2010 lbaldini New module for time conversions added.
    • Added a new utility module handling conversion from met to utc and vice versa. Will be used in the script drawing the orbit in 2d with the SAA overlaid.
    • Relevant Jira(s): GDQMQ-326
  • Common-06-05-03 13-Jan-2010 bregeon more minor change in histogram FastMon/digi comparison.
    • Check GetEntries if TH1, GetSumfOfWeights otherwise that helps for TH1s that do not have the same binning, as SumOfWeights does not count over/under flows in pHistogramPair.py (used by pRootDiffer in order to compare the FastMon and digi eor plots).
  • Common-06-05-02 13-Jan-2010 lbaldini Minor change in histogram FastMon/digi comparison.
    • GetNumEntries() replaced with GetSumOfWeights() in pHistogramPair.py (used by pRootDiffer in order to compare the FastMon and digi eor plots).
  • Common-06-05-01 13-Jan-2010 lbaldini Minor bug fix (> changed in >=).
    • Minor bug fix in alg__values.py: ">" changed to ">=" when checking the branch with the number of entries.
  • Common-06-05-00 12-Jan-2010 lbaldini New min_n parameter implemented for alg__values.
    • alg__values.py modified to support the new min_n parameter, disengaging the alarm for all the trending points obtained with too few events (typically 0).
    • Relevant Jira(s): GDQMQ-334
  • Common-06-04-00 12-Jan-2010 lbaldini Some improvement in the script for the FastMon-digi eor comparison.
    • pHistogramPair.py and pRootDiffer.py modifies in such a way the differences in the histogram binning are handled correctly when comparing the FastMon and digi end-of-run histograms.
    • Relevant Jira(s): GDQMQ-333
  • Common-06-03-00 22-Dec-2009 lbaldini Improvement in the gaussian fitting algorithm.
    • Added a few lines of code to make sure the mean of any gaussian fit lies within the interval in which the fit is performed. This implies a change in pGenericFitAlgorithm.py and a minor refactoring of code in pAlarmBaseAlgorithm.
    • Relevant Jira(s): GDQMQ-329
  • Common-06-02-00 22-Dec-2009 lbaldini Some minor refactoring and improvements in the debug messages.
    • Some minor code refactoring which turned to be handy for a little tool to convert a CAL channel number (in the data monitoring conventions) into its physical location (i.e. tower, layer, etc). No change in any of the functionalities.
    • Some improvements in the debug printouts.
    • Minor change (debug message added) to the pBaseReportGenerator class.

dataMonitoring/FastMon: FastMon-05-01-01

  • FastMon-05-01-01 15-Jan-2010 bregeon fix clock tick deviation calculus and add SAA 2d plot
    • python/pEvtMetaContextProcessor.py bug fix - GDQMQ-331 - calculate clock ticks deviation ony when current AND previous time tones are complete as in Digi EOR
    • python/pEvtMetaContextProcessor.py bug fix for the rollover of the clock tick dev calculation - GDQMQ-325
    • Added script to generate the 2d plot of the orbit with the SAA ovelaid (required some minor modifications to the pSAAPolygon class).

dataMonitoring/IGRF: IGRF-02-00-00

  • IGRF-02-00-00 01-Jan-2010 lbaldini Temporary fix for the 2010 issue.
    • Brute force fix to prevent the package from crashing the pipeline after January 1, 2010 (the field model is only valid up to that date). Will need some work to find a decent fix.
    • Relevant Jira(s): GDQMQ-332
    • Note that this version is already running in the pipeline after an emergency patch.

svac/Monitor: Monitor-01-05-00

  • Monitor-01-05-00 18-Jan-2010 bregeon Handling earth limb correction for rate normalization - GDQMQ-317 GDQMQ-330
    • Add code to use the earth limb correction polynome in RFun NormalizedRate was overloaded to take a new parameter ZenithAngle after the McIlwainL
    • Add code to read EarthLimb correction parameters
    • Relevant Jira(s): GDQMQ-317 GDQMQ-330
  • Monitor-01-04-00 15-Jan-2010 lbaldini Some new variables added.
    • Added new function in RFun.h and RFun.cxx to loop over tracker towers and planes and sum arrays in quandrature (used to sum up errors).
    • Added file doc/johan.readme containing some notes about how to add new quantities to the trending (mostly an e-mail from David Paneque).
    • Added the errors for some quantities with alarms on: trackermon/MonInput_fracSat_err, trackermon/MonInput_layerOcc_err, trackermon/MonInput_stripOcc_err (and MonObjFactory.cxx modified accordingly).
    • Relevant Jira(s): GDQMQ-327
  • Monitor-01-03-04 15-Jan-2010 bregeon Add GoodEvent quantity - GDQMQ-325 v1r3p4 2010/01/15 (DEV)
    • Add new object MonObjInput_GoodEvent to be able to cut on this variable to remove error events from the digi distributions as in FastMon GDQMQ-325

svac/EngineeringModelRoot: v4r4
svac/TestReport: TestReport-10-01-00*

users/richard/pipelineDatasets: v0r6

ft2Util: v1r2p31

evtClassDefs v0r14

GPLtools: GPLtools-01-15-01-fo04

Things that still need to be done.

Feel free to edit/add/remove things (especially remove things if you do something).

  1. Re-tune the alarms on the merit trending normalized rates (Johan and Luca, GDQMQ-317).
  2. Setup the infrastructure for the 2d plot of the orbit with the SAA overlaid (Maria Elena and Max, GDQMQ-326).

Things that need to be checked before going to prod

Please write a comment if you actually check one of the items.

  • Look how corresponding histograms compare in FastMon vs. Digi. They all should look pretty much the same, now. Relevant Jira(s): GDQMQ-325, GDQMQ-333, GDQMQ-325
    • check the output of pRootDiffer.py - only DeltaEventTime_TH1 and DeltaWindowOpen_TH1 may be different.

This is almost right, as DeltaEventTime_TH1 and DeltaWindowOpen_TH1 are indeed different for the run processed in dev. We'll have to pin this down, at some point (LB).

  • Make sure the Merit trending normalized rates look correct (there have been tons of changes). Relevant Jira(s): GDQMQ-310, GDQMQ-317, GDQMQ-330.

This looks ok for the run processed in dev (LB)

  • Make sure the 2d plot of the orbit with the SAA overlaid is correctly created (and, possibly, is linked from the data monitoring page, though this isn't really related to the L1Proc). Relevant Jira(s): GDQMQ-326.
  • Make sure the new errors on the tracker trending quantities that are have alarms on correctly show up. Relevant Jira(s): GDQMQ-327.

This looks ok (LB)

  • Make sure the alarms involving a gaussian fitting do not screw up (the underlying algorithm has slightly changed). Relevant Jira(s): GDQMQ-329.

Looks right, no alarms on the recon eor (LB)

  • Specifically take a look at the output of the alarm on the digi trending quantity Mean_Tick20MHzDeviation: ite is the first one using the new min_n feature of the y_values algorithm. Relevant Jira(s): GDQMQ-331, GDQMQ-334.

Clean (LB)