Recent News

Reasons for Change

• move logs from u15 to u52
• adjust rocking angle cut in gtktime to 52 deg
• adjust process id assignment for weekly PGWave and DRP tasks
• rename createTarBall.py in asp_pgwave and drpMonitoring packages for SCons builds
• handle zero event extractions for GRB afterglow processes

Test Procedure

Tested in dev on data in /ASP/TestSims2 and on relevant data in prod.

Rollback Procedure

Revert to ASP v3r1p0

CCB Jira

ssc-236@jira

Details

• ASP v3r1p1
  • AspHealPix v0r0p3
  • AspLauncher v1r4p1*
    • Modified AspLauncher.xml to use u52 for logs
    • Modified AspInsertIntervals.xml to use u52 for logs
    • Adjusted offset for process id assignments to account for leap seconds in weekly PGWave and DRP_monitoring tasks in launchStreams.py
  • AspPolicy v0r6p1
  • BayesianBlocks v2r0p0
  • asp_pgwave v1r10p3*
    • Modified PGWave.xml to use u52 for logs and to use asp_pgwave_createTarBall.sh
  • drpMonitoring v1r7p6*
    • Modified DRP_monitoring.xml to use u52 for logs and to use drp_createTarBall.sh
    • Changed rocking angle cut from 47 to 52 deg in assignRois.py and getRoiData.py
  • grbASP v4r8p7*
    • Modified GRB_blind_search.xml to use u52 for logs
    • Modified GRB_refinement_launcher.xml to use u52 for logs
    • Modified GRB_refinement.xml to use u52 for logs
    • Modified GRB_afterglow_launcher.xml to use u52 for logs
    • Modified GRB_afterglow.xml to use u52 for logs
    • Added code to makeAfterglowPlots.py to check for zero events in the counts spectrum file.
  • pyASP v3r6p2
  • asp_healpix v2r2p3
  • asp_skymaps v1r13p2
  • asp_pointlike v6r14p1

Reason for change

To move log files from u15 to u52 and to switch to ROOT 5.20.

Test Procedure

We have processed runs in the DEV pipeline with this version of L1Proc.

Rollback procedure

We can easily switch back to the previous version of L1Proc.
CCB Jira

SSC-235@JIRA
Details

L1Pipeline: L1Pipeline v1r80

• Move log files from u15 to u52
• Set retries=3 for processes that access CalibDB

GlastRelease: GlastRelease-15r47p12gr08

I think the only change in this GR wrt the version in use in L1, GR v15r47p12p06 is:

• Switch to ROOT 5.20

dataMonitoring/AlarmsCfg: AlarmsCfg-05-20-02

• reactivate the alarm on the FSW compression level.

calibTkrUtil: v2r9p1

• works with ROOT 5.20

Complete set of tags for L1Proc 1.75
Code Versions
GlastRelease (sim/recon): GlastRelease-v15r47p12gr08*
ScienceTools (Level 2) : v9r15p5
Science Ops (task defs, scripts):
Level 1 pipeline code and applications running in L1:

svac/L1Pipeline: L1Pipeline v1r80*

calibTkrUtil v2r9p1
calibGenTKR v4r5

dataMonitoring/AlarmsCfg: AlarmsCfg-05-20-02*
dataMonitoring/FastMonCfg: FastMonCfg-02-00-01
dataMonitoring/DigiReconCalMeritCfg: DigiReconCalMeritCfg-01-04-03

dataMonitoring/Common: Common-06-01-01
dataMonitoring/FastMon: FastMon-05-01-00
datMonitoring/IGRF: v1r0p1

svac/Monitor: Monitor-01-03-03
svac/EngineeringModelRoot: v4r4
svac/TestReport: TestReport-10-01-00*

users/richard/pipelineDatasets: v0r6

ft2Util: v1r2p31

evtClassDefs v0r14

GPLtools: GPLtools-01-15-01-fo04

Reasons for Change

• Move to ST v9r15p5 from ST v9r15p4
• New version of ASP/BayesianBlocks v2
• Handle empty ROIs that arise from increased rocking angle for six_hour intervals in DRP_monitoring task.
• Use later versions of asp_pointlike, asp_skymaps, asp_healpix packages that were modified for SCons builds
• Rename package-specific clean_up_files.py scripts since SCons builds somehow cannot deploy into python subdirectories.

Test Procedure

Tested in dev on data in /ASP/TestSims2.

Rollback Procedure

Revert to ASP v3r0p3 and ST v9r15p4.

CCB Jira

ssc-234@jira

Details

• ASP v3r1p0
  • AspHealPix v0r0p3
  • AspLauncher v1r4p0
  • AspPolicy v0r6p1
  • BayesianBlocks v2r0p0*
    • Revised interface to allow for ncp_prior to be specified when light_curve method is called rather than in the constructor
  • asp_pgwave v1r10p2*
    • In getPgwInputData.py, refactor logic to select appropriate EVENT_CLASS cuts for test data vs flight data
    • clean_up_files.py -> asp_pgwave_clean_up_files.py
    • modify PGWave.xml to use asp_pgwave_clean_up_files.sh (new xml task def)
  • drpMonitoring v1r7p5*
    • Test each ROI in assignRois.py to ensure events remain after standard gtselect and gtmktime filtering is applied.
    • clean_up_files.py -> drp_clean_up_files.py
    • modify DRP_monitoring.xml to use drp_clean_up_files.py (new xml task def)
  • grbASP v4r8p6*
    • use new BayesianBlocks interface
    • add autoRetryMaxAttempts to GRB_blind_search blind_search process (new xml task def)
    • clean_up_files.py -> grbASP_clean_up_files.py
    • modify GRB_afterglow.xml to use grbASP_clean_up_files.py (new xml task def)
  • pyASP v3r6p2*
    • add pipelineServer() function so that clients can easily determine whether the code is run in 'PROD' or 'DEV'
  • asp_healpix v2r2p3*
  • asp_skymaps v1r13p2*
  • asp_pointlike v6r14p1*

Reason for change

The new version (AlarmsCfg-05-20-03, as opposed to AlarmsCfg-05-20-01) includes a modification to accommodate the new LAT configuration in which the TEM diagnostics is enabled by default.

Test Procedure

We have processed monitoring products from real on-orbit data (LPA) locally with this version of AlarmsCfg.

Rollback procedure

The package can be rolled back to the previous version by flipping a soft link. Also note that the package is completely independent from any other package running in the pipeline and will not cause a version change of L1Proc.

CCB Jira

SSC-233@JIRA

Details (release notes for dataMonitoring/AlarmCfg for AlarmsCfg-05-20-03)

AlarmsCfg-05-20-03 16-Oct-2009 lbaldini Changed limits on the uncompressed/compressed event size)

• Limits on digi_trend OutF_Ratio_EvtSize_CompressedEvtSize changed after the enabling of the diagnostics.

Reason for change

We would like to run for a few days the RHEL3 build of GlastRelease-v15r47p12gr06 (candidate release for the switchover to RHEL4), before moving on to RHEL4.
Plus, we are now Running the ScienceTools from our own installation on afs to improve robustness.

Test Procedure

We have processed runs in the DEV pipeline with this version of L1Proc.

Rollback procedure

We can easily switch back to the previous version of L1Proc.

CCB Jira

SSC-230@JIRA

Details

L1Pipeline: L1Pipeline v1r78
- Use new style tags for monitoring packages.
- FastMon called with the -s option (SAA config).
- Running the ScienceTools from afs.
- Cleaning up the JO for digi/recon.

GlastRelease: GlastRelease-15r47p12gr06

- There are a few changes in this GR wrt the version in use in L1, GR v15r47p12 (mostly implemented to get ready for RHEL4):
- enums v3r0p1: set TEMBUG flag to 0x100000 so it is distinct from PHASEERROR
- Trigger v6r4p1gr2: update test JO; modify handling of TriggerAlg.mask to use Gaudi's StringProperty to avoid any questions from conversion from int to unsigned int
- TkrDigi v2r7p3gr1: Mods to unit test JO
- RootIo v21r12p0gr05: Try making RootConvert public, but no_auto_imports for OnboardFilterTds; adjust req file to make RootConvert private; Patch to allow use of EvtMax==0 to read in a complete input ROOT file
- OnboardFilter v4r14p0gr1: Update to unit test JO
- LdfConverter v4r3p4gr1: mod to use EvtMax==0 to denote reading in a complete input file, to avoid issues with setting EvtMax > MAXINT
- IExternal/Geant4Runtime v3r801p7: Rebuild of g4 to set G4_NO_VERBOSE and G4_NO_STORE_TRAJECTORY, and now includes the tables with the windows distribution
- IExternal/Geant4 v3r801p6: Increment tag so installer recognizes we have a new version of G4
- HepRepSvc v0r27p2gr1: replace abs with fabs in src/FilterFillter.cxx
- HepRepCorba v2r0p0gr1: Pick up some fixes from the HEAD for a new version of L1proc
- Gleam v7r2p3gr1: Patch JO for TriggerAlg.mask setting via string and remove mask setting within ldf2digi.txt JO, as it is unnecessary due to use of ConfigSvc
- GlastSvc v9r26p1gr1: apply updates for random service to apply seed once per run the default
- CalXtalResponse v0r21p8gr1: removed use of obf, no necessary now that RootIo is hiding use of RootConvert
- CalRecon v6r7p0gr1: Modified src/CalEventEnergyAlg.cxx to remove tools Philippe Bruel reminds us are no longer used
- AnalysisNtuple v2r51p0gr03: Remove obsolete CAL merit variables as requested by Philippe Bruel; Fix init of FT1EventClass; Fix for FT1EventClass init

- There are a few changes in the system tests wrt GR v15r47p12. These changes arose from a change in seed handling in GlastSvc to switch from an event to run seed for MC generation.

dataMonitoring/AlarmsCfg: AlarmsCfg-05-20-02
- reactivate the alarm on the FSW compression level.

svac/TestReport: TestReport-10-01-00
(note: this version was approved for L1Proc 1.77 but accidentally not implemented)
- error list is now pointing to the evtGemId.
- checking the FSW compression level (GDQMQ-320).
- checking that periodic evts never have the TEM bug (GDQMQ-319).

Complete set of tags for L1Proc 1.75

Code Versions

GlastRelease (sim/recon): GlastRelease-v15r47p12gr06*

ScienceTools (Level 2) : v9r15p3gl2

Science Ops (task defs, scripts):

Level 1 pipeline code and applications running in L1:

svac/L1Pipeline: L1Pipeline v1r78*

calibTkrUtil v2r7p3
calibGenTKR v4r5

dataMonitoring/AlarmsCfg: AlarmsCfg-05-20-01*
dataMonitoring/FastMonCfg: FastMonCfg-02-00-01
dataMonitoring/DigiReconCalMeritCfg: DigiReconCalMeritCfg-01-04-03

dataMonitoring/Common: Common-06-01-01
dataMonitoring/FastMon: FastMon-05-01-00
datMonitoring/IGRF: v1r0p1

svac/Monitor: Monitor-01-03-03
svac/EngineeringModelRoot: v4r4
svac/TestReport: TestReport-10-01-00*

users/richard/pipelineDatasets: v0r6

ft2Util: v1r2p31

evtClassDefs v0r14

GPLtools: fileOps3

Reason for change

The new version (AlarmsCfg-05-20-01, as opposed to AlarmsCfg-05-19-04) includes some limits to take into account the noisy channel 2414, and the unexpected reboot to FSW 2.0.2, after to EPU0 reboot.

Test Procedure

We have processed monitoring products from real on-orbit data (LPA) locally with this version of AlarmsCfg.

Rollback procedure

The package can be rolled back to the previous version by flipping a soft link. Also note that the package is completely independent from any other package running in the pipeline and will not cause a version change of L1Proc.

CCB Jira

SSC-229@JIRA

Details (release notes for dataMonitoring/AlarmCfg for AlarmsCfg-05-20-01)

AlarmsCfg-05-20-01 02-Oct-2009 bregeon Add temporary alarms because of noisy cal channel 2414

• Add exceptions for channel 2414 for calPedsAnalyzer:
  CalXAdcPedRMS_LEX1_TH1 y_values
  CalXAdcPedRMS_LEX8_TH1 y_values
• Add exceptions for Tower 12 b/C of channel 2414 for digi and fastmon eor:
  CalX_NHit_TH1_Tower_ x_average and x_rms
  CalLogEndRangeHitCounter_Range*_TH2_Tower_12 spikes_and_holes
• Change back lkower limits on digi trending Mean_CompressionLevel changed from
  8.0, 8.0 to 7.0, 7.8 to accommodate with the reboot to the B2-0-2

AlarmsCfg-05-20-00 30-Sep-2009 lbaldini Added alarms on the CAL hitmaps.

• Added alarms on the CAL hitmaps to identify possible spikes and/or holes,
  along the lines of what we already have for the tracker.
  Specifically alarms ("empty_bins" and "spikes_and_holes") have been added,
  with reasobale settings, on:
  CalLogEndRangeHitCounter_Range0_TH2_Tower_*
  CalLogEndRangeHitCounter_Range1_TH2_Tower_*
  CalLogEndRangeHitCounter_Range2_TH2_Tower_*
  CalLogEndRangeHitCounter_Range3_TH2_Tower_*
  See also http://www-glast.stanford.edu/protected/mail/datamon/3435.html and
  the relative thread.

AlarmsCfg-05-19-05 28-Sep-2009 lbaldini Minor limit change for ACD tile 73.

• Upper warning limit for AcdPedPedMeanDeviation_PMTB_TH1 (y_value) changed
  from 8 to 12. This is temporary change that will be reverted as soon as we
  have updated the claibration constants.

• Relevant jira(s): GDM-321

AlarmsCfg-05-19-04 24-Sep-2009 monzani added two checks to verify

• Added two more error conditions to the verify alarm file: checking the FSW
  compression level (GDQMQ-320), checking that periodic evts never have the TEM bug (GDQMQ-319)

Reason for change

We are adding a number of new monitoring tags (mostly, to monitor the distance to the SAA).
Plus, the staging for crumbs is now moved to xrootd.

Test Procedure

We have processed runs in the DEV pipeline with this version of L1Proc.

Rollback procedure

We can easily switch back to the previous version of L1Proc.

CCB Jira

SSC-226@JIRA

Details

L1Pipeline: L1Pipeline v1r77
- Stage crumbs (but not chunks) on xroot.
- Make more smaller crumbs.
- Fail fast if too many chunks.
- Don't make ft2Txt file.

GPLtools: fileOps3
- Support crumbs staging on xroot.

dataMonitoring/AlarmsCfg: AlarmsCfg-05-19-05
- Upper warning limit for AcdPedPedMeanDeviation_PMTB_TH1 (y_value) changed from 8 to 12. This is temporary change that will be reverted as soon as we have updated the claibration constants (GDQMQ-321).
- Added two more error conditions to the verify alarm file: checking the FSW compression level (GDQMQ-320), checking that periodic evts never have the TEM bug (GDQMQ-319).
- Limits on the digi trending quantity Mean_CompressionLevel moved by an "epsilon" as a workaround to a bug in the alarm handler (which is now fixed).
- num_sigma parameter set to zero for the alarm on digi trending quantity Mean_CompressionLevel (since we do not expect any single value lower than 8, we want to explicitely ignore the error on the average).
- Upper limits on digi trending OutF_Ratio_EvtSize_CompressedEvtSize changed from 3.25, 3.50 to 3.40, 3.60 to accommodate the B2-1-0 version of the flight software (error events are now compressed).
- Lower limits on digi trending Mean_CompressionLevel changed from 7.0, 7.8 to 8.0, 8.0 to accommodate the B2-1-0 version of the flight software. Now all the events are compressed and everything whose compression level is different from 8 should be regarded as unexpected.

dataMonitoring/FastMonCfg: FastMonCfg-02-00-01
- Added a new xml file with the definition of the SAA uploaded on the spacecraft (GDQMQ-311).

dataMonitoring/Common: Common-06-01-01
- Some more improment in the calculation of the badness in pAlarmLimits.py. In the completely degenerate case in which all the four limits coincide the badness is no longer set to ERROR_BADNESS + DELTA_BADNESS, as it used to be. Rather, an additional constant, proportional to the distance between the (best) value and the center of the interval (possibly weighted with the error) is added in such a way the sorting (badness-wise) between the data points is preserved (i.e. the they don't have all the same badness, but rather worts is labeled as worst).
- Bug fix in how the alarm limits are handled. The issue was the case in which all the four limits were identical and the actual value was outside the limits themselves. A similar issue was documented and fixed in GDQMQ-272, but I missed one of the cases in the logic (now the code looks perfectly simmetric).

dataMonitoring/FastMon: FastMon-05-01-00
- Test program added to read a m7 file and make plots of the orbit and dstance to the SAA (GDQMQ-311).
- Some more work for the calculation of the distance to the SAA. Still pretty rough, but first reasonable implementation.
- New small module pTETEUtils.py added to the repository to handle the transformation between the J2000 and the True Equator True Equinox (TETE) systems.

svac/Monitor: Monitor-01-03-03
- Added new object to retrieve quantity FastMon_Spacecraft_Distance_To_SAA (GDQMQ-311).
- Added new object to retrieve quantity FT1EventClass (GDQMQ-308).

dataMonitoring/DigiReconCalMeritCfg: DigiReconCalMeritCfg-01-04-03
- Added FastMon trending quantity FastMon_Spacecraft_Distance_To_SAA (GDQMQ-311).

svac/TestReport: TestReport-10-01-00
- error list is now pointing to the evtGemId.
- checking the FSW compression level (GDQMQ-320).
- checking that periodic evts never have the TEM bug (GDQMQ-319).

Complete set of tags for L1Proc 1.75

Code Versions

GlastRelease (sim/recon): v15r47p12

ScienceTools (Level 2) : v9r15p3gl2

Science Ops (task defs, scripts):

Level 1 pipeline code and applications running in L1:

svac/L1Pipeline: L1Pipeline v1r77*

calibTkrUtil v2r7p3
calibGenTKR v4r5

dataMonitoring/AlarmsCfg: AlarmsCfg-05-19-05*
dataMonitoring/FastMonCfg: FastMonCfg-02-00-01*
dataMonitoring/DigiReconCalMeritCfg: DigiReconCalMeritCfg-01-04-03*

dataMonitoring/Common: Common-06-01-01*
dataMonitoring/FastMon: FastMon-05-01-00*
datMonitoring/IGRF: v1r0p1

svac/Monitor: Monitor-01-03-03*
svac/EngineeringModelRoot: v4r4
svac/TestReport: TestReport-10-01-00*

users/richard/pipelineDatasets: v0r6

ft2Util: v1r2p31

evtClassDefs v0r14

GPLtools: fileOps3*

Reason for change

The new version (AlarmsCfg-05-19-04, as opposed to AlarmsCfg-05-18-04) includes some limit changes to accommodate the fact that in the B2-1-0 version of the flight software all the events are compressed.

Test Procedure

We have processed monitoring products from real on-orbit data (LPA) locally with this version of AlarmsCfg.

Rollback procedure

The package can be rolled back to the previous version by flipping a soft link. Also note that the package is completely independent from any other package running in the pipeline and will not cause a version change of L1Proc.

CCB Jira

SSC-225@JIRA

Details (release notes for dataMonitoring/AlarmCfg for AlarmsCfg-05-19-04)

AlarmsCfg-05-19-04 24-Sep-2009 monzani added two checks to verify

• Added two more error conditions to the verify alarm file: checking the FSW compression level (GDQMQ-320), checking that periodic evts never have the TEM bug (GDQMQ-319)

AlarmsCfg-05-19-02 22-Sep-2009 lbaldini Minor change to protect against a bug in the alarm handler

• Limits on the digi trending quantity Mean_CompressionLevel moved by an "epsilon" as a workaround to a bug in the alarm handler (which is now fixed and will make it into the L1Proc at some point, see GDQMQ-272 for details).

AlarmsCfg-05-19-01 22-Sep-2009 lbaldini Minor change in one digi trend alarm.

• num_sigma parameter set to zero for the alarm on digi trending quantity Mean_CompressionLevel (since we do not expect any single value lower than 8, we want to explicitely ignore the error on the average).

AlarmsCfg-05-19-00 21-Sep-2009 lbaldini Digi trend limits updated for flight software B2-1-0

• Upper limits on digi trending OutF_Ratio_EvtSize_CompressedEvtSize changed from 3.25, 3.50 to 3.40, 3.60 to accommodate the B2-1-0 version of the flight software (error events are now compressed).
• Lower limits on digi trending Mean_CompressionLevel changed from 7.0, 7.8 to 8.0, 8.0 to accommodate the B2-1-0 version of the flight software. Now all the events are compressed and everything whose compression level is different from 8 should be regarded as unexpected.

Reasons for Change

• Fixes from ASP v2r12p2 (that had to be rolled back)
• Include ASP versions of pointlike, healpix, skymaps packages from ST v9r11 (later versions cannot be made to work)
• Move to ST v9r15p4 from ST v9r11 (required to handle new EVENT_CLASS content from FT1 switchover on Aug 13)

Test Procedure

Tested in dev on data in /ASP/TestSims2 and on relevant L1Proc datasets in prod

Rollback Procedure

Revert to ASP v2r12p1 and ST v9r11.

CCB Jira

ssc-219@jira

Details

• ASP v3r0p3
  • AspHealPix v0r0p3*
    • use asp_healpix * ASP
  • AspLauncher v1r4p0*
    • Modified insertIntervals.py code to allow flight data to be analyzed in DEV without adding extraneous intervals to the TIMEINTERVALS db table
  • AspPolicy v0r6p1
  • BayesianBlocks v1r1*
    • Modified to use updated ScData interface in Likelihood v15+
  • asp_pgwave v1r10p1*
    • add "ATEL" as valid sourcesub_type for point source selection (from v1r9p5)
    • accommodate interface change in pointlike for localization (from v1r9p5)
    • use asp_pointlike * ASP
    • In refinePositions.py, copy CONVERSION_TYPE column to EVENT_CLASS column in FT1 data for use by asp_pointlike code.
  • drpMonitoring v1r7p4*
    • Add W Comae to public release sources
    • update sourceType attribute in XML model definition using db table sourcesub_typ info
  • grbASP v4r8p4*
    • use reply_to feature for sending email alerts from BlindSearch task (from v4r8p2)
    • skip unreliable FERMI_LAT_POSITION notices in GRB_refinement task (from v4r8p2)
    • bug-fix for handling single event FT1 files for far off-axis bursts (from v4r8p2)
    • adjust formatting of GCN Notice to accommodate changes in Bacodine ingest system
  • pyASP v3r6p1*
    • add reply_to option for MulitPartMailer class
  • asp_healpix v2r2p2*
  • asp_skymaps v1r13p1*
  • asp_pointlike v6r14*

Reason for change

This is the L1 that we will use for the public data release.

Because of an issue - still not understood - with CalCfpEnergy and Root 5.20 in Glast Release, we have decided to stick with Root 5.18 for the moment. This means that we had to take Science Tools v9r15p3 - which we need and which is compiled against Root 5.20 because of important bug fixes to TMinuit2 - and compile it against Root 5.18.00c-gl1. This has become ST v9r15p3gl2. The reason Science Tools and Glast Release are connected in such a way for L1 is because of ft2Util that links against both ST and GR (it reads in digi and merit files and writes out FT2 files). We have gone through the dependencies to make sure that nothing L1 uses from ST uses TMinuit2 (i.e. would need the bug fixes in Root 5.20) so ST v9r15p3gl2 should be safe to use for L1.

Test Procedure

We have processed run 271088335 in the DEV pipeline with this version of L1Proc and compared it to the original L1 processing. The files are:

root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/1.75/ft1/gll_ph_r0271088335_v003.fit
root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/1.75/ft2/gll_pt_r0271088335_v003.fit
root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/1.75/ls1/gll_ev_r0271088335_v003.fit
root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/1.75/ls3/gll_lt_r0271088335_v003.fit
root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/1.75/merit/r0271088335_v003_merit.root
root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/1.75/filteredMerit/r0271088335_v002_filteredMerit.root

We have validated the data in the following ways:

- CalCfpEnergy, beyond the Root 5.20 issue, also seems to produce slightly different results depending on the computer architecture. The SLAC batch farm contains several different ones, in particular rhel3 and rhel4-64, in addition to Intel and AMD. A run is always split into hundreds of pieces that randomly run on all of these different machines. This makes an event-by-event comparison more difficult. It also occasionally changes the number of events in the FT1 file. In this case the number of events in the FT1 file decreased from 36685 to 36682 from the original processing to the new one.

- I have verified for the test run that all FT1 columns except for ENERGY are identical between the two processings. For the ENERGY column, I see 49 events with a difference. All but two are tiny, <<1%. While two changes are at the 10% and 20% level. Neither of the two were Diffuse photons. This is consistent with changes previously seen with different computer architectures.

- Note that the CalCfpEnergy issue that has just been discovered is not understood at all and all comparisons should be seen in that light. It's clear that all data processed with the current GR (v15r47p12) - which we are not updating in the L1 release - suffers from this feature. And it's not clear when it was first introduced. Also note that this CalCfpEnergy issue seems to predominantly affect background events. When there is a change for Diffuse photons it's at the level of 10^-6 relative energy variation. This probably indicates that the problem is related to unstable/sensitive fits and background events. But that is just a guess.

- We have verified that FT1 EVENT_CLASS in the new processing is identical to FT1 CTBCLASSLEVEL in the original processing.

- We have verified through spot checks that the new FT1 Diffuse response columns are the same as the ones in Benoit's 11 months sample. Columns DIFRSP0 and DIFRSP1 in our FT1 files are the same as DIFRSP0 and DIFRSP2 in Benoit's file (there are tiny numerical differences between DIFRSP1 and DIFRPS2 - presumably round off errors). For this comparison we needed an older run and used run 263204954. The reprocessed FT1 file is here: root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/1.75/ft1/gll_ph_r0263204954_v001.fit. Benoit's file was /nfs/farm/g/glast/u33/lott/11month_dataset/ft1_may_rock43.fits.

- We have verified that the following columns are identical in the two FT2 files: START,STOP,LAT_GEO,LON_GEO,RAD_GEO,RA_ZENITH,DEC_ZENITH,B_MCILWAIN,L_MCILWAIN, GEOMAG_LAT,RA_SCZ,DEC_SCZ,RA_SCX,DEC_SCX,LAT_MODE,LIVETIME,QSJ_1,QSJ_2,QSJ_3,QSJ_4. The SC_POSITION column has to be expanded and is not covered by the diff script. I made a few spot checks. No changes.

- We have verified that the following new FT2 columns are present:: RA_NPOLE, DEC_NPOLE, ROCK_ANGLE, LAT_CONFIG, DATA_QUAL

- We have done spot checks to verify the consistency between LS1 and the original Merit ntuple.

Rollback procedure

We could switch back to the previous version of L1Proc. In that case we would have to turn off exporting files to the FSSC and switch to a mode where we do continous reprocessing for the FSSC files. Note that the current L1 pipeline uses the same Glast Release as this new one.

CCB Jira

SSC-216@JIRA

Details

L1Pipeline v1r75:
- Make ft1 on chunks.
- Add diffuse response to ft1.
- Make our own gap file.
- Add quality and config to ft2.
- Creating filtered Merit files (photons only, for now).
- Move a bunch of stuff from setSuccessful to setRunning.
- Allow specifying the version to fileNames.fileName.
- Add version to chunk file names (but it's always 0 at the mo).
- Tweak hsplit to work around ROOT 5.20 issue.
- Allow data catalog type to differ from group.
- Allow attaching extra metadata when registering files - use to specify cut for filtered merit.
- fileNames.fileName can get file name from a variable, if so it overrides the normal rules.

scienceTools: v9r15p3gl2
- This was a major upgrade for Science Tools - the old L1 was still at v9r8p2.
- L1 is only using makeFT1, gtltcube, gtmktime and gtdiffrsp from the ScienceTools.
- In addition, ft2Utils and Verify are linking to Science Tools..
- This special version of Science Tools is compiled against Root 5.18.00c-gl1. See the top of the page for additional details. Note that the 'gl2' in the ST version number is not related to the Root version it is based on, but is purely a cvs branch tag number.

evtClassDef: v0r14p00
- v0r6p1: added updated LS1 dictionary file from Julie
- v0r7: add cuts and classifier script for Pass 7
- v0r8: add (pass-through) P7_electron_Classifier.py
- v0r9: add classifier script to map CTBClassLevel to EVENT_CLASS for Pass 6 reprocessing
- v0r10: new LS1variables
- v0r11: removed 5 new LS1variables (per Julie)
- v0r12: removed 'return' chars from LS1variables
- v0r13: revert to LS1variables of v0r9 (per Julie) to be used for first public data release
- v0r14: update Pass6_Reprocessing_Classifier

ft2Util: v1r2p31
- new version handling both old and new ft2 tpl file
- force DigiStart and DigiStop from command line
- added LiveTimeTolerance parameter to set to zero values below the tolerance
- fixed bug in mergeft2.cxx concerning ra_sc* values
- zero-length entries at the end of the 1-second ft2file are checked and skipped
- Fixes merging bug for IN_SAA (was previously not defined).

Complete set of tags for L1Proc 1.75

Code Versions

GlastRelease (sim/recon): v15r47p12

ScienceTools (Level 2) : v9r15p3gl2*

Science Ops (task defs, scripts):

Level 1 pipeline code and applications running in L1:

svac/L1Pipeline: v1r75*

calibTkrUtil v2r7p3
calibGenTKR v4r5

dataMonitoring/AlarmsCfg: v5r17p4
dataMonitoring/FastMonCfg: v2r0p0
dataMonitoring/DigiReconCalMeritCfg: v1r4p1

dataMonitoring/Common: v6r0p0
dataMonitoring/FastMon: v5r0p0
datMonitoring/IGRF: v1r0p1

svac/Monitor: v1r2p37
svac/EngineeringModelRoot: v4r4
svac/TestReport: v9r2

users/richard/pipelineDatasets: v0r6

ft2Util: v1r2p31*

evtClassDefs v0r14*

GPLtools: fileOps1

Reason for change

The database tables have been modified to allow users to associate:

• LAT catalog sources to ASP point sources (in order to fetch light curve data)
• ASP point sources with other ASP point sources (the same object has been named differently from different groups)
• LAT catalog sources with other LAT catalog sources (when a new catalog is released the names change)

The changes to the tables AND the web applications are backward compatible and transparent to the user.

Test Procedure

This version has been tested on the DEV server.

Rollback procedure

Version 3.3 could be reinstalled

CCB Jira

SSC-215@JIRA

Reason for change

The new version (v5r18p3, as opposed to v5r18p2) includes some minor limit changes to protect a few alarms when the statistics is not enough.

Test Procedure

We have processed monitoring products from real on-orbit data (LPA) locally with this version of AlarmsCfg.

Rollback procedure

The package can be rolled back to the previous version by flipping a soft link. Also note that the package is completely independent from any other package running in the pipeline and will not cause a version change of L1Proc.

CCB Jira

SSC-214@JIRA

Details (release notes for dataMonitoring/AlarmCfg for v5r18p3)

v5r18p3
A few more alarms protected against low statistics.

• Recon eor:
  • ReconAcdPhaMipsCorrectedAngle_PMTA_TH1_AcdTile_* and ReconAcdPhaMipsCorrectedAngle_PMTB_TH1_AcdTile_*: condition on min entries changed from 10 to 35.
• Digi eor:
  • CondArrCNOTrigger_TH1, algorithms x_average, x_rms, low_high_ratio, y_values and reference_histogram: condition on min entries changed from 10000 to 15000.
• FastMon eor:
  • CondArrCNOTrigger_TH1, algorithms x_average, x_rms, low_high_ratio, y_values and reference_histogram: condition on min entries changed from 10000 to 15000.

Reason for change

The new version (v5r18p2, as opposed to v5r17p4) includes some minor limit changes to protect a few alarms when the statistics is not enough.

Test Procedure

We have processed monitoring products from real on-orbit data (LPA) locally with this version of AlarmsCfg.

Rollback procedure

The package can be rolled back to the previous version by flipping a soft link. Also note that the package is completely independent from any other package running in the pipeline and will not cause a version change of L1Proc.

CCB Jira

SSC-213@JIRA

Details (release notes for dataMonitoring/AlarmCfg for v5r18p2)

v5r18p2
A few more alarms protected against low statistics.

• Digi eor:
  • CondArrCNO_CalLoOpensWindow_TH1: added condition on min entries (1000) for algorithms x_average, x_rms and low_high_ratio, reference_histogram.
  • CondArrTKR_CalLoOpensWindow_TH1: added condition on min entries (20000) for reference histogram.

• FastMon eor:
  • CondArrCNO_CalLoOpensWindow_TH1: added condition on min entries (1000) for algorithms x_average, x_rms and low_high_ratio, reference_histogram.
  • CondArrTKR_CalLoOpensWindow_TH1: added condition on min entries (20000) forreference histogram.

v5r18p1
A few more alarms protected against low statistics.

• Digi eor:
  • CalX_NHit_TH1_Tower_*: min entries 100000 -> 500000 for algorithm x_rms.
  • CalX_Total_NHit_TH1: min entries 100000 -> 500000 for algorithm low_high_ratio.
  • CondArrCalHiTrigger_TH1: added condition on min entries (60000) for algorithms x_average, x_rms and low_high_ratio.
  • CondArrTKR_CalLoOpensWindow_TH1: added/changed condition on min entries (20000) for algorithms x_average, x_rms and low_high_ratio.
  • TkrHits_TH1_Tower_*: added condition on min entries (500000) for algorithm x_average.

• FastMon eor:
  • CalX_NHit_TH1_Tower_*: min entries 100000 -> 500000 for algorithm x_rms.
  • CalX_Total_NHit_TH1: min entries 100000 -> 500000 for algorithm low_high_ratio.
  • CondArrCalHiTrigger_TH1: added condition on min entries (60000)for algorithms x_average, x_rms and low_high_ratio.
  • CondArrTKR_CalLoOpensWindow_TH1: added/changed condition on min entries (20000) for algorithms x_average, x_rms and low_high_ratio.
  • TkrHits_TH1_Tower_*: added condition on min entries (500000) for algorithm x_average.

v5r18p0

• Upper warning limit changed from 3.0 to 3.5 for CAL channel 2784, calpeds_eor alarms CalXAdcPedPedMeanDeviation_LEX1_TH1 and CalXAdcPedPedMeanDeviation_HEX1_TH1.
• Separate alarm limits for CAL channel 2785 on calpeds_eor alarm CalXAdcPedRMS_HEX1_TH1 (0.6, 0.8 -> 0.9, 1.2)

• Relevant jira(s): GDQMQ-315

Reason for change

This new GR, v15r47p12, contains an improved error message in case of event errors and also removes an incorrect warning, repeated 160 times per event, in case we have data with TEM diagnostics. Most importantly, it fixes a bug with the ldfReader which prevented the TEM diagnostic from being interpreted correctly. We anticipate that taking data with the TEM diagnostics will be the new default running mode when the new FSW B2-1-0 is uploaded and activated. There is no change to any functionality in this GlastRelease.

We have also added a few new monitoring quantities related to event sizes at Eric Siskind's request, plus a new variable that monitors the distance from the SAA.

Finally, we are getting rid of the chunk-based host list (it didn't obviously help but produced a lot of trouble when some configuration changes were applied to LFS).

Test Procedure

We have processed data runs in the DEV pipeline with this version of L1Proc.

Rollback procedure

We can switch back to the previous version of L1Proc.

CCB Jira

SSC-212@JIRA

Details

L1Pipeline v1r74:
- Getting rid of the chunk-based hostlist.

GlastRelease v15r47p12:
- There are a few changes in this GR wrt the version in use in L1, GR v15r47p7:
- ldfReader v7r4p1: windows patch to use const_iterator versus iterator; patch to set diagnostic and error bits properly by taking OR across contributions and making sure all diagnostic data is stored, even those without associated CAL or TKR data; patch check on diagnostic size
- RootIo v21r12gr1: patch to return kTRUE in GleamMessageHandler::Notify
- LdfConverter: patch to set various levels for EbfDebug
- EventIntegrity v0r8p2: Print out the error flag (JIRA GRINF-51) and event id when errors are found.

- There are no changes in the system tests wrt GR v15r47p7.

svac/TestReport: v9r2
- Completeness job option modified (was inconsistent with L1 config)

dataMonitoring/DigiReconCalMeritCfg: v1r4p1
- Added quantities to address JIRA GDQM-309.

dataMonitoring/FastMonCfg: v2r0p0
- Added a new variable (distance to the SAA) in the baseConfig.xml file (JIRA GDQMQ-311).
dataMonitoring/Common: v6r0p0
- Added -s option to pOptionParser to pass to the xml file with the SAA definition to dataMonitoring/FastMon/pDataProcessor.py (necessary for the plot of the distance to the SAA, as in JIRA GDQMQ-311).
dataMonitoring/FastMon: v5r0p0
- Quite a few files modified to accomodate the plot of the distance to the SAA (JIRA GDQMQ-311). The detailed description of the changes is:

• pDataProcessor.py: one more command line option added (-s) to pass the xml file with the definition of the SAA.
• pGeomagProcessor.py: added filling of the output tree with the distance to the SAA variable.
• pM7Parser.py: new class member (the SAA polygon) added.
• pSAAPolygon.py: EARTH_RADIUS not imported anymore from pSCPosition due to an import clash.
• pSCPosition.py: added a new class member (the SAA polygon) and a new method returning the distance of a generic point from the polygon itself.

Complete set of tags for L1Proc 1.74

Code Versions

GlastRelease (sim/recon): v15r47p12*

ScienceTools (Level 2) : v9r8p2

Science Ops (task defs, scripts):

Level 1 pipeline code and applications running in L1:

svac/L1Pipeline: v1r74*

calibTkrUtil v2r7p3
calibGenTKR v4r5

dataMonitoring/AlarmsCfg: v5r17p4
dataMonitoring/FastMonCfg: v2r0p0*

dataMonitoring/DigiReconCalMeritCfg: v1r4p1*

dataMonitoring/Common: v6r0p0*
dataMonitoring/FastMon: v5r0p0*
datMonitoring/IGRF: v1r0p1

svac/Monitor: v1r2p37
svac/EngineeringModelRoot: v4r4
svac/TestReport: v9r2*

users/richard/pipelineDatasets: v0r6

ft2Util: v1r2p23

evtClassDefs v0r6

GPLtools: fileOps1

Reason for change

The new version (v5r17p4, as opposed to v5r16p2) introduces a few minor limit changes and protections against low statistics detailed below.

Test Procedure

We have processed monitoring products from real on-orbit data (LPA) locally with this version of AlarmsCfg.

Rollback procedure

The package can be rolled back to the previous version by flipping a soft link. Also note that the package is completely independent from any other package running in the pipeline and will not cause a version change of L1Proc.

CCB Jira

SSC-211@JIRA

Details (release notes for dataMonitoring/AlarmCfg for v5r17p4)

v5r17p4
Quite a few alarms protected against low statistics.

• Digi eor:
  • AcdVetoMap_TH1 (x_rms): added condition on minimum number of entries (100000).
  • Acd_GemVeto_AcdTile_TH1 (x_rms): added condition on minimum number of entries (100000).
  • CondArrCNO_CalLoOpensWindow_TH1 (x_rms): added condition on minimum number of entries (100).
  • CondArrCNO_TKROpensWindow_TH1 (x_rms): added condition on minimum number of entries (200).
  • TkrHits_TH1_Tower_* (x_average): added condition on minimum number of entries (10000).
  • CondSummaryWord_TH1 (reference_histogram): added condition on minimum number of entries (10000).
  • CondArrCalLo_TKROpensWindow_TH1 (reference_histogram): added condition on minimum number of entries (5000).
  • CondArrTKR_CalLoOpensWindow_TH1 (reference_histogram): added condition on minimum number of entries (1000).
• FastMon eor:
  • CondArrCalLo_TKROpensWindow_TH1 (reference_histogram): added condition on minimum number of entries (5000).
  • CondArrTKR_CalLoOpensWindow_TH1 (reference_histogram): added condition on minimum number of entries (1000).
  • CondSummaryWord_TH1 (reference_histogram): added condition on minimum number of entries (10000).
  • CondArrCNO_CalLoOpensWindow_TH1 (x_rms): added condition on minimum number of entries (100).
  • CondArrCNO_TKROpensWindow_TH1 (x_rms): added condition on minimum number of entries (200).

v5r17p3

• Alarm limits changed for CondArrCNO_CalLoOpensWindow_TH1 in both digi and fastmon eor.
  • warning max: 0.0006 -> 0.0008
  • error max : 0.0010 -> 0.0015

v5r17p2

• Condition on the minimum number of entries changed from 10 to 15 in CondArrCalHi_CNOOpensWindow_TH1 for the alarms on the average and the rms of the distribution.

v5r17p1

• Warning high limit on CondSummaryWord_TH1 (reference histogram) for both digi end of run and fastmon end of run changed from 100 to 110 (see http://www-glast.stanford.edu/protected/mail/datamon/2892.html)

v5r17p0

• Warning upper limit for OutF_Ratio_EvtSize_CompressedEvtSize in digi_trend changed from 3.2 to 3.25.