Reason for change
We have a new GlastRelease that uses FSW B1-1-0 and which is compatible with the new DFI/evt file format. This release is needed to process data taken with the new FSW build that will be activated on Thursday July 31.
There are also general updates to the OBF filters to improve the agreement between the FSW and OBF Gamma-filters, a change to AcdRecon to catch G4 propagator problems, a change to the L1 job options that caused TKR calib crashes under some circumstances and a fix so that ACD tiles that are hit are displayed in FRED.
We have also re-enabled the "crash-in-case-of-Moot-key-mismatch" feature which was disabled in the Emergency request update to L1Proc 1.61.
There are also a lot of updates to the Data Monitoring and the Alarms, bug fixes and improvements to the L1 pipeline itself and a fix that will enable the Skimmer to ingest merged Root files produced by the L1 pipeline.
Test Procedure
FSW and Bryson provided evt files made with FSW 1-1-0 and in the new format so we could verify that GlastRelease could process the new data. Testing in the DEV pipeline has been made difficult by the fact that this data did not exist in Moot or in the Acquisition tables, did not have a corresponding Magic 7 file, dependet on ISOC_TEST etc. We have worked around most of these problems, for example by disabling Moot access for the testing. This means that we will have to make minor modifications to some of the scripts and job options before we put this L1Proc into PROD and that we have not been able to test everything together at the same time in the DEV pipeline. Usually we test the exact code.
Rollback procedure
L1Proc can be NOT be rolled back to the previous version if we activate FSW B1-1-0 on the instrument.
CCB Jira
Details
L1Pipeline v1r62
- Fixed the dependencies for copyM7Hp and registerM7Hp and copyM7Lci.
- Make chunkLists human-editable. LONE-93@JIRA
- Test for pathological chunks. LONE-92@JIRA
- Avoid core files in log directory from ingest jobs.
- Don't fail kludgeAsp if the stream has already been launched. ASP-26@JIRA
- Put runId and chunkId in target when logging missing files on merge. LONE-94@JIRA
- Trim a couple of static values from registerPrep/registerStuff interface.
- Require 50GB free scratch for merging recon.
- Performance/reliability tweaks. LONE-100@JIRA
- Change the way batch jobs are run so the ISOC environment can be chosen dynamically.
- In test mode only, acqQuery.runTimes returns bogus hardcoded values if run is not in ACQSUMMARY.
GlastRelease v15r33:
- Includes a working GCRCalib.
- Systemtestsfor GR v15r33with respect to the previous production release, GR v15r24. The differences are understood.
- RM diffwith respect to GR v15r24.
svac/EngineeringModelRoot: v4r1p7
- Added state variables for all the OBF filters and version number for the FSW Gamma-filter.
dataMonitoring/AlarmsCfg: v2r4p2
- Limits adjusted for digi trending. Relevant variables: Rate_AcdGemCNO_GARC, Mean_TkrTotalHitsPerEvt, OutF_NormalizedTkrTriggerWithLessThan6Layers_Tower, Rate_NTkrHits_TowerPlane, Mean_ConsecutivePlanesHit_Tower, DoubleDiffRate_LiveTimeFraction.
- New start_x parameter also used for the CAL LO trigger threshold plot.
- Recon eor improved taking advantage of the new start_x parameter of the alg__leftmost_edge algorithm (used for the alarms on the CAL LAC thresholds and the ACD veto thresholds, for which the threshold on the significance has also been changed from 60 to 20).
- Got rid of a couple of alarms on unstable recon quantities.
- Limits on the merit trending and recon eor updated.
- A few limits updated based on the new runs.
- A few limits updated based on the new runs.
- Got rid of the alarm on the rms of TkrHits_TH1_Tower (Digi).
- A few limits updated based on the new runs.
- Got rid of the alarm on the rms of TkrHits_TH1_Tower (FastMon).
- Added an exception for the hot strip in tower 3.
dataMonitoring/FastMonCfg: v1r4p4
- Add ToT plots per tkr tower layer controller. Relevant jira(s): GDQMQ-148
- Fixed all cuts to get rid of events with an error (error_summary<64) in config.xml. I choose not to propagate this cut to the configLCI since there should NOT be error in LCI runs, and if there is we want to spot them by any means. Relevant jira(s): GDQMQ-203
- Fixed all cuts to be explicitly booleans in config.xml and configLCI.xml. Relevant jira(s): GDQMQ-208
- Added AcdHitChannel to monitor the ACD occupancy per cable, channel in config.xml and configLCI.xml
- Corresponding plot name is AcdHitsCounter_CableChannel_TH2. Relevant jira(s): GDQMQ-1
- New error_summary variable added. Relevant jira(s): GDQMQ-173
- Move new_second and clocktics_dev_20MHz from all secondary xml files to baseConfig. Relevant jira(s): GDQMQ-196
- Added Z axis pointing in galactic coordinates L,B in baseConfig.xml. Relevant jira(s): GDQMQ-193
dataMonitoring/DigiReconCalMeritCfg: v1r2p10
- But corrected for quantity Rate_NTkrHits_TowerPlane. This addresses Jira GDQMQ-221
- Correct errors of some quantities: Jira GDQMQ-218.
- Add quantities to solve jira GDQMQ-214
- Add quantities to solve jira GDQMQ-200
- Solve issue described in Jira GDQMQ-207.
- New quantities (jiras GDQMQ-201 and GDQMQ-211) and bug correction (jira GDQMQ-213)
- FastMon trending of Z axis pointing in galactic coordinates. GDQMQ-198
dataMonitoring/Common: v3r7p0
- Possibility to select asymmetric fit ranges for the Cal Ped/Gain and the Acd Ped analyzers implemented.
- Add the correct error bars to the output histograms of the Cal Ped/Gain and the Acd Ped analyzers.
- Added the distribution of the mean, rms and reduced chisquare to the output root file from the Cal Ped/Gain and the Acd Ped analyzers.
- Minor improvements to the plots appearance for the Cal Ped/Gain and the Acd Ped analyzers.
- Added a switch to the Cal gain analyzer to get rid of the highest bin in the left/right plots (unphysical spikes). Relevant jira(s): GDQMQ-215
- New feature implemented in the leftmost_edge algorithm. Now the sliding window loop does not necessarily start from the first bin. A generic x value can be set from the xml configuration file. This improves the speed a lot and potentially makes the algorithms more stable. Relevant jira(s): GDQMQ-210
- Minor bug fix in alg__values and test function removed as it was broken.
- Exceptions taken into account when setting the output value in alg__spikes_and_holes
- Exception violations put back in. Relevant jira(s): GDQMQ-188
- Update to pGlobal, added the number of channels per ACD cable. Relevant jira(s): GDQMQ-1
- Minor improvement: errors not displayed for trending plots with no associated errors (used to look as +- 0.000).
- Minor aesthetics change in pAlarmBaseAlgorithm---the algorithm names were mispelled.
- Different way to associate the branch arrays to the relative errors in alg__values; now it's more general and gracefully handles Hiro's trackermon quantities---though it was not crashing before and gave the same number. It essentially suppresses a ROOT warning.
- Cast to float in pAlarmiLimits.getBadness() added to prevent numpy to return inf upon ZeroDivision error. Relevant jira(s): GDQMQ-206
dataMonitoring/FastMon: v3r5p6
- pErrorEvent : Move the TIMETONE errors to the lowest bits so that it is easier not to cut them out but cutting real errors applyingError_summary < 64 - GDQMQ-173
- pError : Reorganize error dictionnary to take into account new errors reported by the new version of the LDF. For now, I do not store the details of the errors reported by the different subsystems, besides the TEM_BUG. Relevant jira(s): GDQMQ-173, GDQMQ-197
- Minor improvements in the interface to the output of pErrorEvent.
- More informations added to the error summary xml and some internal improvements.
- pFastMonReportGenerator updated accordingly.
- Merging script modified to accomodate the changes.
- Bug fix in pSCPosition Ra axis pointing calculation, folding in 2Pi was not done correctly Relevant jira(s): GDQMQ-212
- Added a plot to monitor the ACD occupancy in electronic space AcdHitsCounter_CableChannel_TH2 give the occupancy per ACD cable, channel AcdHitChannel[12][18] is the associated tree variable. Relevant jira(s): GDQMQ-1
- New error_summary variable added (new FastMonCfg v1r4p0). Relevant jira(s): GDQMQ-173
- Cable tags added to the details of the phasing errors.
- Improved xml output for the error handler.
- Error documentation moved from the old text file to the python class file.
- Error summary parsing added.
- New error_summary variable added. Relevant jira(s): GDQMQ-197, GDQMQ-173
- Event iterators changed to detect instances of the TEM firmware bug and not to care of unphysical strips anymore. Relevant jira(s): GDQMQ-197
svac/Monitor: v1r2p16
- Modified function RFun::computeratio to check for channel saturation before usage. This addresses (partly) Jira GDQMQ-215.
- New quantities added to address issue from jiras GDQMQ-214
- New quantities added to address issues from jiras GDQMQ-201 and GDQMQ-211.
- 'FastMon trending of Z axis pointing in galactic coordinates - Jira GDQMQ-198.
- Change in treemerge.cxx to address Jira GDQMQ-191.
svac/TestReport: v6r15
- Abort is accepted as a valid closing reason instead of Pause. Jira FSW-1082
users/richard/pipelineDatasets: v0r5
- Build an index on ("m_runId", "m_eventId") after merge in pruneTuple::prune. LONE-105@JIRA
GPLtools: v1r11
- Clean up a corner-case crash in stagefiles cleanup, and try to avoid it in the first place. LONE-97@JIRA
- Try really hard to avoid creating directories that already exist. LONE-96@JIRA
Complete set of tags for L1Proc 1.62
Code Versions
GlastRelease (sim/recon) v15r33*
ScienceTools (Level 2) : v9r6p2
Science Ops (task defs, scripts):
Level 1 pipeline code and applications running in L1:
svac/L1Pipeline: v1r62*
calibTkrUtil v2r4p1
calibGenTKR v4r5
dataMonitoring/AlarmsCfg: v2r4p2*
dataMonitoring/FastMonCfg: v1r4p4*
dataMonitoring/DigiReconCalMeritCfg: v1r2p10*
dataMonitoring/Common: v3r7p0*
dataMonitoring/FastMon: v3r5p6*
datMonitoring/IGRF: v1r0p1
svac/Monitor: v1r2p16*
svac/EngineeringModelRoot: v4r1p7*
svac/TestReport: v6r15*
users/richard/pipelineDatasets: v0r5*
ft2Util: v1r2p23
evtClassDefs v0r6
GPLtools: v1r11*