Reason for change
We have a turbo version of CalRecon that increases the overall recon processing speed by a factor two! This GR also includes the correct job options to enable the offline ACD High Range and Coherent Noise calibrations, JIRAS SSC-113 and SSC-112, has fixes for G4 propagator crashes and includes the FSW compressed event size and FSW compression level in the digi file. Note that the ACD trapzoids and Toby's space craft alignment are not in this GR.
FastMon has been upgraded to use the new version of LDF that has been installed in ISOC_PROD. This version has a much improved error parsing, in particular for phasing errors/TEM bug. Note that GR still uses the old version of LDF. Is not affected by this as it gets LDF through GLAST_EXTERNAL. GR will catch up at a later stage.
This L1Proc also has general updates and bug fixes to the monitoring.
Test Procedure
We have processed data runs in the DEV pipeline with this version of L1Proc.
Rollback procedure
We can switch back to the previous version of L1Proc in case of problems.
CCB Jira
Details
L1Pipeline v1r63
- Add alarms on calHist. LONE-107@JIRA
- In test mode only, acqQuery.runTimes returns values supplied by halfpipe if run is not in ACQSUMMARY.
- Always use prod L0 archive to make magic7.
- Automatically determine host groups attached to glastdataq when making host lists.
- Premake run, chunk and crumb directories on buffers to reduce concurrent writes.
- Fix chunk overlap testing, add more tests. LONE-92@JIRA
- Put CMTPATH in environment for all jobs. Might help with broken headers when merging.
GlastRelease v15r39:
- Systemtestsfor GR v15r39with respect to the previous production release, GR v15r33. There are a certain number of small differences in the ACD and CAL. They all seem consistent with random fluctuations and/or the code changes. The one worry is that all the 'random' fluctuations only seem to occur for ACD and CAL quantities, not TKR. However, the differences are small. And it's not obvious how to disentangle these things without making many new releases were we back out of one thing at the time. This is the drawback when we hold of GR for a while and get a lot of new things in all at once.
- RM diffwith respect to GR v15r33.
svac/EngineeringModelRoot: v4r3
- Added compressed event size and compression level.
- Added an explicit exit(1) in case of problems writing the output file, JIRA SVAC-117.
- Added the new Gleam event flag.
dataMonitoring/AlarmsCfg: v4r0p0
- Format of the exceptions donfig file modified as to make the mechanism more flexible. NOTE: the chahnge is not backward compatible. Need dataMonitoring/Common v4r1p0.
- Upper warning limit for spikes_and_holes on TkrHitsCounter_PlaneGTFE_TH2_Tower_ moved from 400 to 1000 (we were having quite a few of not significant warnings).
- Number of sigma for issuing warnings/errors moved from 1 to 2 on all the tkrTrend alarms, as per Hiro's request.
- Alarm limits on the calhist monitoring products made wider so that they do not cause spurious alarms in the tuning phase.
- Alarms on the ACD asymmetries added.
- Bins not corresponding to unphysical ACD tiles removed from the alarms (uses a new ad-hoc feature of the y_values alarm algorithm).
- Added a few alarms on the ACD pedestals.
- num_sigma set to zero for all the alarms on calhist; This takes care, for the moment, of the wrong error bars on the plots.
- Lower error limits on Mean_ToT_con0_Trunc_1_249_TowerPlane (digi trend) changed from 15 to 1 in such a way that an error is reported only if a controller is dead (and not in case of noisy strips). High error limits moved to 150.
- Upper warning limit on Rate_NTkrHits_TowerPlane moved from 2.2 to 2.5 in digi trend. Relevant jira(s): GDQMQ-237
dataMonitoring/FastMonCfg: v1r4p5
- Add ToT plots per plane in merging list file : MergeHistos_FastMon.txt Relevant jira(s): GDQMQ-148
dataMonitoring/DigiReconCalMeritCfg: v1r2p17
- Add to address Jira GDQMQ-232.
- Add quantities to address jiras GDQMQ-224 and GDQMQ-239. Change cut FswFilters[0]>0 for FswGammaStatus ==0 in merit configs.
- Corrected bug i monconfig_recon_histos.xml (related to jira GDQMQ-229)
- Corrected quantities addressed by Jiras GDQMQ-229 and GDQMQ-233
- Add quantities addressed by Jira GDQMQ-241.
- Add to address Jira GDQMQ-232
- Changes to address jira GDQMQ-242.
dataMonitoring/Common: v4r1p0
- Changed the format for the xml files for the alarm exceptions. The new format is more flexible and allows to explicitely decide the output status (either WARNING or ERROR) upon exception violation. NOTE: the change is not backward compatible with the old configuration file format. Need datamonitoring/AlarmsCfg v4r4p0.
- Some refactoring in the base class for the alarm algorithm. A new general method called getFitOutput() has been implemented to replace the old getFitParameter() and getFitParametes(). It makes easier to implement new alarms involving fitting.
- Added new base class pGenericFitAlgorithm to deal in a uniform fashion with all the algorithms involving simple fitting.
- alg_gauss_mean, alggauss_rms, alggauss_norm, algpeak_position and alg_peak_width modified as to use the new method and the new base class.
- pBaseAnalyzer class modifies as to use the new method.
- Implemented a mechanism to link to each alarm on a multidimentional trending quantity the actual plot that triggered the alarm. Relevant jira(s): GDQMQ-240
- Added an option to skip a defined set of histogram bins in alg__y_values.py. Necessary to put alarms on the ACD histograms with the GEM tile id on the x axis, since we do have bins which do not correspond to physical tiles, there. Relevant jira(s): GDQMQ-238
- One more handle implemented for the "values" and the "y_values" algorithms: multiplicative factor on the error bar of each trending point/bin value that can be set from the xml file (corresponds to put an alarm at the level of n sigma). Defaults to one, which was the previous behaviour. This also allows to put a zero factor which is handy for the digiLong histograms, in
which the errors are wrong, at the moment.
- Minimum TrueTimeInterval for the alg_values increased from 5 to 10 s. Relevant jira(s): GDQMQ-226
- Added a utility script to map the alarms on the plots.
svac/Monitor: v1r2p23
- This corrects a bug related to the filling of the histograms Garc:Gafe: Jira GDQMQ-244
- Add quantities to address Jira GDQMQ-232.
- Add quantities to address jiras GDQMQ-224 and GDQMQ-239.
- Add function computeratio_v2 and add new quantities. This addresses Jira GDQMQ-241.
- Upgrade of treemerge.cxx. It now Discard chunks contained in between two chunks. This addresses Jira GDQMQ-223
- Add quantities to address Jira GDQMQ-232.
- Add possibiliity of seeting histogram errors to zero. This addresses Jira GDQMQ-242.
dataMonitoring/FastMon: v4r1p3
- Add back cmt requirements file. It had disappeared from the latest tag
- pCustomPlotter tagged with the ACD plot in elx space. It had disappeared from the latest tag
- Looks like an old version of pXmlInputList was tagged. I fixed it and tagged again.
- Retag things below after removing a wrong tag. Hopefully, everything is OK now. I hate cvs.
- Work only with LDF > 6.4.0
- Improve handling of errors following the new version of the LDF, all kind of errors now taken into account : pAEMcontributionIteratorBase.py
- Use the logger to dump debug message in case of an error : pCALcontributionIteratorBase.py
- Improve handling of errors following the new version of the LDF, all kind of errors now taken into account : pEBFeventIterator.py
- Improve handling of errors following the new version of the LDF, all kind of errors now taken into account. TEM_BUG is now known and reported as such : pERRcontributionIteratorBase.py
- New error contributions added for each iterators. ACD renamed as AEM : pError.py
- New error bits added. ACD renamed as AEM. LAT_DATAGRAM and EBF_EVENT have the same code : pErrorEvent.py
- Improve handling of errors following the new version of the LDF, all kind of errors now taken into account. Note that the new LDF code checks the status of the iterator and exits whenever a serious error is found. pLATcomponentIterator.py
- Improve handling of errors following the new version of the LDF, all kind of errors now taken into account : pLATcontributionIterator.py
- Use the logger to dump debug message in case of an error : pLATdatagramIterator.py
- Improve handling of errors following the new version of the LDF, all kind of errors now taken into account. EM_BUG is now known and reported as such : pTKRcontributionIteratorBase.py. Relevant jira(s): GDQMQ-197
svac/EngineeringModelRoot: v4r3
- Added EventGleamFlags.
- Added compressed event size and compression level. Needs GR v15r35 or greater. Added 'exit(1)' in case of problems writing the svac.root file.
- Added version of the gamma-filter to the ntuple.
svac/TestReport: v6r16
- Enabling only needed branches in digi takes care of memory leaks
users/richard/pipelineDatasets: v0r6
- Log result of header->Write in pruneTuple::copyHeader.
Complete set of tags for L1Proc 1.63
Code Versions
GlastRelease (sim/recon) v15r39*
ScienceTools (Level 2) : v9r6p2
Science Ops (task defs, scripts):
Level 1 pipeline code and applications running in L1:
svac/L1Pipeline: v1r63*
calibTkrUtil v2r5
calibGenTKR v4r5
dataMonitoring/AlarmsCfg: v4r0p0*
dataMonitoring/FastMonCfg: v1r4p5*
dataMonitoring/DigiReconCalMeritCfg: v1r2p17*
dataMonitoring/Common: v4r1p0*
dataMonitoring/FastMon: v4r1p3*
datMonitoring/IGRF: v1r0p1
svac/Monitor: v1r2p23*
svac/EngineeringModelRoot: v4r3*
svac/TestReport: v6r16*
users/richard/pipelineDatasets: v0r6*
ft2Util: v1r2p23
evtClassDefs v0r6
GPLtools: v1r11