Blog from September, 2013

Reason for change

We request to put AlarmsCfg-06-01-07 in production. It features a single minor tweak to the alarm limits to cope with a known problematic ACD channel.

The detailed description of the changes is in the last section.

Test Procedure

We have processed monitoring products from real on-orbit data (LPA) locally with this version of AlarmsCfg.

Rollback procedure

The package can be rolled back to the previous version by flipping a soft link. Also note that the package is completely independent from any other package running in the pipeline and will not cause a version change of L1Proc.

CCB Jira

SSC-360@JIRA

Details (release notes for AlarmsCfg-06-01-07)

We are reprocessing run 351769427 several times, with different settings.

L1Proc 3.0-PROD, GR 17-35-24-gr15, ST 09-24-00, calib flavor vanilla

L1Proc stream: http://glast-ground.slac.stanford.edu/Pipeline-II/exp/Fermi/si.jsp?stream=76321789&dataSourceMode=Prod
First log file (digi): /nfs/farm/g/glast/u41/L1/logs/PROD/L1Proc/3.0/doRun/doChunk/digitization/120xxxxxx/225xxx/001/351xxxxxx/769xxx/427/096xxx/180/logFile.txt
Digi job options (in order of inclusion, NOT in order of execution):
- Digitization: /afs/slac.stanford.edu/g/glast/ground/releases/volume14/GlastRelease-17-35-24-gr15/jobOptions/Gleam/jobOptions/pipeline/ldf2digi.txt
- Calibrations: /afs/slac.stanford.edu/g/glast/ground/releases/volume14/GlastRelease-17-35-24-gr15/jobOptions/Gleam/jobOptions/pipeline/calibrations.txt
- Geometry: /afs/slac.stanford.edu/g/glast/ground/releases/volume14/GlastRelease-17-35-24-gr15/xml/xmlGeoDbs/flight/flightSegVols.xml
- Pipeline: /afs/slac.stanford.edu/g/glast/ground/PipelineConfig/Level1/3.0/data/digi.jobOpt
First log file (recon): /nfs/farm/g/glast/u41/L1/logs/PROD/L1Proc/3.0/doRun/doChunk/doCrumb/recon/120xxxxxx/225xxx/001/351xxxxxx/769xxx/427/096xxx/180/000/logFile.txt
Recon job options (in order of inclusion, NOT in order of execution):
- Reconstruction: /afs/slac.stanford.edu/g/glast/ground/releases/volume14/GlastRelease-17-35-24-gr15/jobOptions/Gleam/jobOptions/pipeline/readigi_runrecon.txt
- Calibrations: /afs/slac.stanford.edu/g/glast/ground/releases/volume14/GlastRelease-17-35-24-gr15/jobOptions/Gleam/jobOptions/pipeline/calibrations.txt
- Geometry: /afs/slac.stanford.edu/g/glast/ground/releases/volume14/GlastRelease-17-35-24-gr15/xml/xmlGeoDbs/flight/flightSegVols.xml
- Pipeline: /afs/slac.stanford.edu/g/glast/ground/PipelineConfig/Level1/3.0/data/recon.jobOpt
Output files:
- Digi: root://glast-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/prod/3.0/digi/r0351769427_v000_digi.root
- Merit: root://glast-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/prod/3.0/merit/r0351769427_v000_merit.root
- Recon: root://glast-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/prod/3.0/recon/r0351769427_v000_recon.root
- FT1: root://glast-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/prod/3.0/ft1/gll_ph_p120_r0351769427_v000.fit

L1Proc 4.4-DEV, run 351769427, GR 17-35-24-lp61, ST 09-32-05, calib flavor L1current

L1Proc stream: http://glast-ground.slac.stanford.edu/Pipeline-II/exp/Fermi/si.jsp?stream=2326556&dataSourceMode=Dev
First log file (digi): /nfs/farm/g/glast/u41/L1/logs/DEV/L1Proc/4.4/doRun/doChunk/digitization/120xxxxxx/225xxx/001/351xxxxxx/769xxx/427/096xxx/180/logFile.txt
Digi job options (in order of inclusion, NOT in order of execution):
- Digitization: /afs/slac.stanford.edu/g/glast/ground/releases/volume10/GlastRelease-17-35-24-lp61/jobOptions/Gleam/jobOptions/pipeline/ldf2digi.txt
- Calibrations: /afs/slac.stanford.edu/g/glast/ground/releases/volume10/GlastRelease-17-35-24-lp61/jobOptions/Gleam/jobOptions/pipeline/calibrations.txt
- Geometry: /afs/slac.stanford.edu/g/glast/ground/releases/volume10/GlastRelease-17-35-24-lp61/xml/xmlGeoDbs/flight01/flightSegVols.xml
- Pipeline: /afs/slac.stanford.edu/g/glast/ground/PipelineConfig/Level1/4.4/data/digi.jobOpt
First log file (recon): /nfs/farm/g/glast/u41/L1/logs/DEV/L1Proc/4.4/doRun/doChunk/doCrumb/recon/120xxxxxx/225xxx/001/351xxxxxx/769xxx/427/096xxx/180/000/logFile.txt
Recon job options (in order of inclusion, NOT in order of execution):
- Reconstruction: /afs/slac.stanford.edu/g/glast/ground/releases/volume10/GlastRelease-17-35-24-lp61/jobOptions/Gleam/jobOptions/pipeline/readigi_runrecon.txt
- Calibrations: /afs/slac.stanford.edu/g/glast/ground/releases/volume10/GlastRelease-17-35-24-lp61/jobOptions/Gleam/jobOptions/pipeline/calibrations.txt
- Geometry: /afs/slac.stanford.edu/g/glast/ground/releases/volume10/GlastRelease-17-35-24-lp61/xml/xmlGeoDbs/flight01/flightSegVols.xml
- Pipeline: /afs/slac.stanford.edu/g/glast/ground/PipelineConfig/Level1/4.4/data/recon.jobOpt
Output files:
- Digi: root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/4.4/digi/r0351769427_v021_digi.root
- Merit: root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/4.4/merit/r0351769427_v021_merit.root
- Recon: root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/4.4/recon/r0351769427_v017_recon.root
- FT1: root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/4.4/ft1/gll_ph_p202_r0351769427_v020.fit

L1Proc 4.4-DEV, run 388740704, GR 17-35-24-lp61, ST 09-32-05, calib flavor L1current

L1Proc stream: http://glast-ground.slac.stanford.edu/Pipeline-II/exp/Fermi/si.jsp?stream=2326755&dataSourceMode=Dev
First log file (digi): /nfs/farm/g/glast/u41/L1/logs/DEV/L1Proc/4.4/doRun/doChunk/digitization/388xxxxxx/740xxx/704/388xxxxxx/740xxx/704/116xxx/232/logFile.txt
Digi job options (in order of inclusion, NOT in order of execution):
- Digitization: /afs/slac.stanford.edu/g/glast/ground/releases/volume10/GlastRelease-17-35-24-lp61/jobOptions/Gleam/jobOptions/pipeline/ldf2digi.txt
- Calibrations: /afs/slac.stanford.edu/g/glast/ground/releases/volume10/GlastRelease-17-35-24-lp61/jobOptions/Gleam/jobOptions/pipeline/calibrations.txt
- Geometry: /afs/slac.stanford.edu/g/glast/ground/releases/volume10/GlastRelease-17-35-24-lp61/xml/xmlGeoDbs/flight01/flightSegVols.xml
- Pipeline: /afs/slac.stanford.edu/g/glast/ground/PipelineConfig/Level1/4.4/data/digi.jobOpt
First log file (recon): /nfs/farm/g/glast/u41/L1/logs/DEV/L1Proc/4.4/doRun/doChunk/doCrumb/recon/388xxxxxx/740xxx/704/388xxxxxx/740xxx/704/116xxx/232/000/logFile.txt
Recon job options (in order of inclusion, NOT in order of execution):
- Reconstruction: /afs/slac.stanford.edu/g/glast/ground/releases/volume10/GlastRelease-17-35-24-lp61/jobOptions/Gleam/jobOptions/pipeline/readigi_runrecon.txt
- Calibrations: /afs/slac.stanford.edu/g/glast/ground/releases/volume10/GlastRelease-17-35-24-lp61/jobOptions/Gleam/jobOptions/pipeline/calibrations.txt
- Geometry: /afs/slac.stanford.edu/g/glast/ground/releases/volume10/GlastRelease-17-35-24-lp61/xml/xmlGeoDbs/flight01/flightSegVols.xml
- Pipeline: /afs/slac.stanford.edu/g/glast/ground/PipelineConfig/Level1/4.4/data/recon.jobOpt
Output files:
- Digi: root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/4.4/digi/r0388740704_v001_digi.root
- Merit: root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/4.4/merit/r0388740704_v001_merit.root
- Recon: root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/4.4/recon/r0388740704_v001_recon.root
- FT1: root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/4.4/ft1/gll_ph_p202_r0388740704_v001.fit

L1Proc_a 4.6-DEV, GR 17-35-24-lp60, ST 09-32-05, calib flavor L1current

L1Proc stream: http://glast-ground.slac.stanford.edu/Pipeline-II/exp/Fermi/si.jsp?stream=2318938&dataSourceMode=Dev
First log file (digi): /nfs/farm/g/glast/u41/L1/logs/DEV/L1Proc_a/4.6/doRun/doChunk/digitization/120xxxxxx/225xxx/001/351xxxxxx/769xxx/427/096xxx/180/logFile.txt
Digi job options (in order of inclusion, NOT in order of execution):
- Digitization: /afs/slac.stanford.edu/g/glast/ground/releases/volume10/GlastRelease-17-35-24-lp60/jobOptions/Gleam/jobOptions/pipeline/ldf2digi_a.txt
- Calibrations: /afs/slac.stanford.edu/g/glast/ground/releases/volume10/GlastRelease-17-35-24-lp60/jobOptions/Gleam/jobOptions/pipeline/calibrations_a.txt
- Geometry: /afs/slac.stanford.edu/g/glast/ground/releases/volume10/GlastRelease-17-35-24-lp60/xml/xmlGeoDbs/flight01/flightSegVols.xml
- Pipeline: /afs/slac.stanford.edu/g/glast/ground/PipelineConfig/Level1/4.6/data/digi.jobOpt_a
First log file (recon): /nfs/farm/g/glast/u41/L1/logs/DEV/L1Proc_a/4.6/doRun/doChunk/doCrumb/recon/120xxxxxx/225xxx/001/351xxxxxx/769xxx/427/096xxx/180/000/logFile.txt
Recon job options (in order of inclusion, NOT in order of execution):
- Reconstruction: /afs/slac.stanford.edu/g/glast/ground/releases/volume10/GlastRelease-17-35-24-lp60/jobOptions/Gleam/jobOptions/pipeline/readigi_runrecon_a.txt
- Calibrations: /afs/slac.stanford.edu/g/glast/ground/releases/volume10/GlastRelease-17-35-24-lp60/jobOptions/Gleam/jobOptions/pipeline/calibrations_a.txt
- Geometry: /afs/slac.stanford.edu/g/glast/ground/releases/volume10/GlastRelease-17-35-24-lp60/xml/xmlGeoDbs/flight01/flightSegVols.xml
- Pipeline: /afs/slac.stanford.edu/g/glast/ground/PipelineConfig/Level1/4.6/data/recon.jobOpt_a
Output files:
- Digi: root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/4.6/digi/r0351769427_v020_digi.root
- Merit: root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/4.6/merit/r0351769427_v020_merit.root
- Recon: root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/4.6/recon/r0351769427_v016_recon.root
- FT1: root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/4.6/ft1/gll_ph_p202_r0351769427_v019.fit

L1Proc_b 4.6-DEV, run 388740704, GR 17-35-24-lp60, ST 09-32-05, calib flavor (selected overrides): p7repro

L1Proc stream: http://glast-ground.slac.stanford.edu/Pipeline-II/exp/Fermi/si.jsp?stream=2322147&dataSourceMode=Dev
First log file (digi): /nfs/farm/g/glast/u41/L1/logs/DEV/L1Proc_b/4.6/doRun/doChunk/digitization/388xxxxxx/740xxx/704/388xxxxxx/740xxx/704/116xxx/232/logFile.txt
Digi job options (in order of inclusion, NOT in order of execution):
- Digitization: /afs/slac.stanford.edu/g/glast/ground/releases/volume10/GlastRelease-17-35-24-lp60/jobOptions/Gleam/jobOptions/pipeline/ldf2digi_b.txt
- Calibrations: /afs/slac.stanford.edu/g/glast/ground/releases/volume10/GlastRelease-17-35-24-lp60/jobOptions/Gleam/jobOptions/pipeline/calibrations_b.txt
- Geometry: /afs/slac.stanford.edu/g/glast/ground/releases/volume10/GlastRelease-17-35-24-lp60/xml/xmlGeoDbs/flight/flightSegVols.xml
- Pipeline: /afs/slac.stanford.edu/g/glast/ground/PipelineConfig/Level1/4.6/data/digi.jobOpt_b
First log file (recon): /nfs/farm/g/glast/u41/L1/logs/DEV/L1Proc_b/4.6/doRun/doChunk/doCrumb/recon/388xxxxxx/740xxx/704/388xxxxxx/740xxx/704/116xxx/232/000/logFile.txt
Recon job options (in order of inclusion, NOT in order of execution):
- Reconstruction: /afs/slac.stanford.edu/g/glast/ground/releases/volume10/GlastRelease-17-35-24-lp60/jobOptions/Gleam/jobOptions/pipeline/readigi_runrecon_b.txt
- Calibrations: /afs/slac.stanford.edu/g/glast/ground/releases/volume10/GlastRelease-17-35-24-lp60/jobOptions/Gleam/jobOptions/pipeline/calibrations_b.txt
- Geometry: /afs/slac.stanford.edu/g/glast/ground/releases/volume10/GlastRelease-17-35-24-lp60/xml/xmlGeoDbs/flight01/flightSegVols.xml
- Pipeline: /afs/slac.stanford.edu/g/glast/ground/PipelineConfig/Level1/4.6/data/recon.jobOpt_b
Output files:
- Digi: root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/4.6/digi/r0388740704_v000_digi.root
- Merit: root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/4.6/merit/r0388740704_v000_merit.root
- Recon: root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/4.6/recon/r0388740704_v000_recon.root
- FT1: root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/4.6/ft1/gll_ph_p202_r0388740704_v000.fit

L1Proc_a 4.4-DEV, GR 17-35-24-lp57, ST 09-32-05, calib flavor L1current

L1Proc stream: http://glast-ground.slac.stanford.edu/Pipeline-II/exp/Fermi/si.jsp?stream=2293138&dataSourceMode=Dev
First log file (digi): /nfs/farm/g/glast/u41/L1/logs/DEV/L1Proc_a/4.4/doRun/doChunk/digitization/120xxxxxx/225xxx/001/351xxxxxx/769xxx/427/096xxx/180/logFile.txt
Digi job options (in order of inclusion, NOT in order of execution):
- Digitization: /afs/slac.stanford.edu/g/glast/ground/releases/volume10/GlastRelease-17-35-24-lp57/jobOptions/Gleam/jobOptions/pipeline/ldf2digi_a.txt
- Calibrations: /afs/slac.stanford.edu/g/glast/ground/releases/volume10/GlastRelease-17-35-24-lp57/jobOptions/Gleam/jobOptions/pipeline/calibrations_a.txt
- Geometry: /afs/slac.stanford.edu/g/glast/ground/releases/volume10/GlastRelease-17-35-24-lp57/xml/xmlGeoDbs/flight01/flightSegVols.xml
- Pipeline: /afs/slac.stanford.edu/g/glast/ground/PipelineConfig/Level1/4.4/data/digi.jobOpt_a
First log file (recon): /nfs/farm/g/glast/u41/L1/logs/DEV/L1Proc_a/4.4/doRun/doChunk/doCrumb/recon/120xxxxxx/225xxx/001/351xxxxxx/769xxx/427/096xxx/180/000/logFile.txt
Recon job options (in order of inclusion, NOT in order of execution):
- Reconstruction: /afs/slac.stanford.edu/g/glast/ground/releases/volume10/GlastRelease-17-35-24-lp57/jobOptions/Gleam/jobOptions/pipeline/readigi_runrecon_a.txt
- Calibrations: /afs/slac.stanford.edu/g/glast/ground/releases/volume10/GlastRelease-17-35-24-lp57/jobOptions/Gleam/jobOptions/pipeline/calibrations_a.txt
- Geometry: /afs/slac.stanford.edu/g/glast/ground/releases/volume10/GlastRelease-17-35-24-lp57/xml/xmlGeoDbs/flight01/flightSegVols.xml
- Pipeline: /afs/slac.stanford.edu/g/glast/ground/PipelineConfig/Level1/4.4/data/recon.jobOpt_a
Output files:
- Digi: root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/4.4/digi/r0351769427_v014_digi.root
- Merit: root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/4.4/merit/r0351769427_v014_merit.root
- Recon: root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/4.4/recon/r0351769427_v011_recon.root
- FT1: root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/4.4/ft1/gll_ph_p202_r0351769427_v014.fit

L1Proc_b 4.4-DEV, GR 17-35-24-lp57, ST 09-32-05, calib flavor (selected overrides): p7repro

L1Proc stream: http://glast-ground.slac.stanford.edu/Pipeline-II/exp/Fermi/si.jsp?stream=2296311&dataSourceMode=Dev
First log file (digi): /nfs/farm/g/glast/u41/L1/logs/DEV/L1Proc_b/4.4/doRun/doChunk/digitization/120xxxxxx/225xxx/001/351xxxxxx/769xxx/427/096xxx/180/logFile.txt
Digi job options (in order of inclusion, NOT in order of execution):
- Digitization: /afs/slac.stanford.edu/g/glast/ground/releases/volume10/GlastRelease-17-35-24-lp57/jobOptions/Gleam/jobOptions/pipeline/ldf2digi_b.txt
- Calibrations: /afs/slac.stanford.edu/g/glast/ground/releases/volume10/GlastRelease-17-35-24-lp57/jobOptions/Gleam/jobOptions/pipeline/calibrations_b.txt
- Geometry: /afs/slac.stanford.edu/g/glast/ground/releases/volume10/GlastRelease-17-35-24-lp57/xml/xmlGeoDbs/flight/flightSegVols.xml
- Pipeline: /afs/slac.stanford.edu/g/glast/ground/PipelineConfig/Level1/4.4/data/digi.jobOpt_b
First log file (recon): /nfs/farm/g/glast/u41/L1/logs/DEV/L1Proc_b/4.4/doRun/doChunk/doCrumb/recon/120xxxxxx/225xxx/001/351xxxxxx/769xxx/427/096xxx/180/000/logFile.txt
Recon job options (in order of inclusion, NOT in order of execution):
- Reconstruction: /afs/slac.stanford.edu/g/glast/ground/releases/volume10/GlastRelease-17-35-24-lp57/jobOptions/Gleam/jobOptions/pipeline/readigi_runrecon_b.txt
- Calibrations: /afs/slac.stanford.edu/g/glast/ground/releases/volume10/GlastRelease-17-35-24-lp57/jobOptions/Gleam/jobOptions/pipeline/calibrations_b.txt
- Geometry: /afs/slac.stanford.edu/g/glast/ground/releases/volume10/GlastRelease-17-35-24-lp57/xml/xmlGeoDbs/flight01/flightSegVols.xml
- Pipeline: /afs/slac.stanford.edu/g/glast/ground/PipelineConfig/Level1/4.4/data/recon.jobOpt_b
Output files:
- Digi: root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/4.4/digi/r0351769427_v015_digi.root
- Merit: root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/4.4/merit/r0351769427_v015_merit.root
- Recon: root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/4.4/recon/r0351769427_v012_recon.root
- FT1: root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/4.4/ft1/gll_ph_p202_r0351769427_v015.fit

L1Proc_a 4.5-DEV, GR 17-35-24-gr35*, ST 09-32-05, calib flavor L1current

L1Proc stream: http://glast-ground.slac.stanford.edu/Pipeline-II/exp/Fermi/si.jsp?stream=2306040&dataSourceMode=Dev
First log file (digi): /nfs/farm/g/glast/u41/L1/logs/DEV/L1Proc_a/4.5/doRun/doChunk/digitization/120xxxxxx/225xxx/001/351xxxxxx/769xxx/427/096xxx/180/logFile.txt
Digi job options (in order of inclusion, NOT in order of execution):
- Digitization: /afs/slac.stanford.edu/g/glast/ground/releases/volume07/GlastRelease-17-35-24-gr35/jobOptions/Gleam/jobOptions/pipeline/ldf2digi_a.txt
- Calibrations: /afs/slac.stanford.edu/g/glast/ground/releases/volume07/GlastRelease-17-35-24-gr35/jobOptions/Gleam/jobOptions/pipeline/calibrations_a.txt
- Geometry: /afs/slac.stanford.edu/g/glast/ground/releases/volume07/GlastRelease-17-35-24-gr35/xml/xmlGeoDbs/flight01/flightSegVols.xml
- Pipeline: /afs/slac.stanford.edu/g/glast/ground/PipelineConfig/Level1/4.5/data/digi.jobOpt_a
First log file (recon): /nfs/farm/g/glast/u41/L1/logs/DEV/L1Proc_a/4.5/doRun/doChunk/doCrumb/recon/120xxxxxx/225xxx/001/351xxxxxx/769xxx/427/096xxx/180/000/logFile.txt
Recon job options (in order of inclusion, NOT in order of execution):
- Reconstruction: /afs/slac.stanford.edu/g/glast/ground/releases/volume07/GlastRelease-17-35-24-gr35/jobOptions/Gleam/jobOptions/pipeline/readigi_runrecon_a.txt
- Calibrations: /afs/slac.stanford.edu/g/glast/ground/releases/volume07/GlastRelease-17-35-24-gr35/jobOptions/Gleam/jobOptions/pipeline/calibrations_a.txt
- Geometry: /afs/slac.stanford.edu/g/glast/ground/releases/volume07/GlastRelease-17-35-24-gr35/xml/xmlGeoDbs/flight01/flightSegVols.xml
- Pipeline: /afs/slac.stanford.edu/g/glast/ground/PipelineConfig/Level1/4.5/data/recon.jobOpt_a
Output files:
- Digi: root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/4.5/digi/r0351769427_v018_digi.root
- Merit: root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/4.5/merit/r0351769427_v018_merit.root
- Recon: root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/4.5/recon/r0351769427_v014_recon.root
- FT1: root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/4.5/ft1/gll_ph_p202_r0351769427_v017.fit
* Note: 17-35-24-gr35 is the oldest GR which can be compiled with rhel5 and therefore tested in the current L1Proc setup.

L1Proc_b 4.5-DEV, GR 17-35-24-gr35*, ST 09-32-05, calib flavor (selected overrides): p7repro

L1Proc stream: http://glast-ground.slac.stanford.edu/Pipeline-II/exp/Fermi/si.jsp?stream=2315944&dataSourceMode=Dev
First log file (digi): /nfs/farm/g/glast/u41/L1/logs/DEV/L1Proc_b/4.5/doRun/doChunk/digitization/120xxxxxx/225xxx/001/351xxxxxx/769xxx/427/096xxx/180/logFile.txt
Digi job options (in order of inclusion, NOT in order of execution):
- Digitization: /afs/slac.stanford.edu/g/glast/ground/releases/volume07/GlastRelease-17-35-24-gr35/jobOptions/Gleam/jobOptions/pipeline/ldf2digi_b.txt
- Calibrations: /afs/slac.stanford.edu/g/glast/ground/releases/volume07/GlastRelease-17-35-24-gr35/jobOptions/Gleam/jobOptions/pipeline/calibrations_b.txt
- Geometry: /afs/slac.stanford.edu/g/glast/ground/releases/volume07/GlastRelease-17-35-24-gr35/xml/xmlGeoDbs/flight/flightSegVols.xml
- Pipeline: /afs/slac.stanford.edu/g/glast/ground/PipelineConfig/Level1/4.5/data/digi.jobOpt_b
First log file (recon): /nfs/farm/g/glast/u41/L1/logs/DEV/L1Proc_b/4.5/doRun/doChunk/doCrumb/recon/120xxxxxx/225xxx/001/351xxxxxx/769xxx/427/096xxx/180/000/logFile.txt
Recon job options (in order of inclusion, NOT in order of execution):
- Reconstruction: /afs/slac.stanford.edu/g/glast/ground/releases/volume07/GlastRelease-17-35-24-gr35/jobOptions/Gleam/jobOptions/pipeline/readigi_runrecon_b.txt
- Calibrations: /afs/slac.stanford.edu/g/glast/ground/releases/volume07/GlastRelease-17-35-24-gr35/jobOptions/Gleam/jobOptions/pipeline/calibrations_b.txt
- Geometry: /afs/slac.stanford.edu/g/glast/ground/releases/volume07/GlastRelease-17-35-24-gr35/xml/xmlGeoDbs/flight01/flightSegVols.xml
- Pipeline: /afs/slac.stanford.edu/g/glast/ground/PipelineConfig/Level1/4.5/data/recon.jobOpt_b
Output files:
- Digi: root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/4.5/digi/r0351769427_v019_digi.root
- Merit: root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/4.5/merit/r0351769427_v019_merit.root
- Recon: root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/4.5/recon/r0351769427_v015_recon.root
- FT1: root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/4.5/ft1/gll_ph_p202_r0351769427_v018.fit
* Note: 17-35-24-gr35 is the oldest GR which can be compiled with rhel5 and therefore tested in the current L1Proc setup.

P202-ROOT, P7REP reprocessing pipeline task, GR 17-35-24-gr17, calibration flavor (selected overrides): p7repro

P202-ROOT, run 388740704, GR 17-35-24-gr17, calibration flavor (selected overrides): p7repro

Summary of the flavors for P202 and the new L1

P202

All calibs are set to vanilla, except for the following eight, which are set to p7repro:

TKR_HotChan, _DeadChan, _TowerAlign, _InternalAlign
CAL_Ped, _MevPerDac, _Asym
ACD_Ped, _ElecGain, _HighRange, _CoherentNoise
NAS_LATAlignment

The corresponding JO:

TkrReconCalib.calibFlavor = "vanilla";
TkrReconCalib.hotStripsCalibFlavor = "p7repro";
TkrReconCalib.deadStripsCalibFlavor = "p7repro";
TkrReconCalib.towerAlignmentCalibFlavor = "p7repro";
TkrReconCalib.internalAlignmentCalibFlavor = "p7repro";

Note: in some tasks that start with digitization, "TkrReconCalib" will be called "TkrCalibAlg".

CalCalibSvc.DefaultFlavor = "vanilla";
CalCalibSvc.FlavorPed = "p7repro";
CalCalibSvc.FlavorMevPerDac = "p7repro";
CalCalibSvc.FlavorAsym = "p7repro";

AcdCalibSvc.DefaultFlavor = "vanilla";
AcdCalibSvc.Ped = "p7repro";
AcdCalibSvc.Gain = "p7repro";
AcdCalibSvc.HighRange = "p7repro";
AcdCalibSvc.FlavorCoherentNoise = "p7repro";

New L1

All calibrations use flavor L1current, and the JO is:

TkrReconCalib.calibFlavor = "L1current";
CalCalibSvc.DefaultFlavor = "L1current";
AcdCalibSvc.DefaultFlavor = "L1current";

Comparison of merit files (Philippe)

Here are some comparisons between the two merit files (confirming Carmelo's initial findings):

The nb of tracks can be very different. When looking at the ang separation between the tkr dir of the two files or the cal dir of the two files, it can be very large (when log10(ang sep)<-3, it is due to round off). The difference in tkr direction is not correlated to the difference in cal direction.

Test with MC (Philippe)

I've produced some MC (AG w/o overlay with 2.5<logE<3.5 and -1.0<ZDir<-0.5) with lp40 and lp56, using exactly the same job options and calibrations.

Here are the comparison plots:

There is no difference (within round off). So the issue we have comes from either calibration or job options.

Comparison of merit files (Carmelo)

Just a few more info, for my first analysis:
Cal centroid differences in X and Y are larger when xtals are all in the same layer, in particular the top and the bottom layer (no difference on Z, only 7 events have different CalZEcntr):

Looking at high-level variables like CalTransRms: 63013 evts differ (~2%), mostly in case of very low CalTransRms ( e.g.56269 have log10(CalTransRms)<-2 ).

TkrNumTracks: 78547 evts differ. Here a scatter plot of these events (Z axis in log scale), mostly +-1 track (still strange), but sometime there are huge differences.

Calibration Comparison (Joanne)

I compared calibration-related output from P202-ROOT (the excerpt Tom included in his email) and the sample 4.4 log file M.E. suggested above. The collection of calibrations (comparing filepaths) is almost identical, however the order in which they are fetched is not, which could indicate the processing steps using those calibrations are not occurring in the same order either, or (looking more likely) that the files I compared were not processing the same data.  Different events can cause calibrations to be fetched in a different order.  

I checked the dates on all calibration files used.  They are all as old as they should be; that is, there is no indication that any of them have been modified since they were first put into use.

The one difference is in splits.  For p202 the file used is $(LATCalibRoot)/TKR/TkrSplits_default.xml

For 4.4 it's $(LATCalibRoot)/TKR/TkrSplits_flight.xml

(Carmelo) The difference between default and flight is that in the latter bottom layer in tower 10 (x=2, y=2) is read out from one end only. Not sure how this info is used in reconstruction, but 'Hit' variables in MeritTuple (like TkrNumHits, TkrHitsInLyrNN, etc ) are identical in the 2 cases. The problems comes only when recon is involved.

Comparison of recon files (Sasha)

I've printed the crystal energies and positions for 11 events with big difference in energy centroid position between pass7rep (attached) and level1new (attached), and I can confirm that energies and positions reconstructed in individual crystals are identical for these two files, while X and Y centroid positions have dramatic difference.

(Carmelo) Thanks Sasha! This means that the problem is not in calibration or rounding, we are somehow using a different algorithm.

Notice that the values in centroid calculation correspond to the energy-weighted average using all xtals in level1new and only some of the xtals in P202.

Comparison of MeritTuple for L1Proc_a 4.6-DEV test (Carmelo)

I made the usual comparison of number of events with different value in MeritTuple wrt P202. The fix introduced in this release improves a lot the TKR part, but not enough to declare victory: there are still many (13427) events with different TKR direction. In the majority of these (96%) CAL cntr and axis is identical to p202.

Here a few summary plots:
I also tried to count the event that change class: 177 out of 2665218 change classification (out of diagonal in table below):

L1\P202

Transient

Source

Clean

UltraClean

Other

Total

Transient

77567

5

2

2

74

77650

Source

3

5217

0

1

1

5222

Clean

1

0

1422

2

0

1425

UltraClean

1

2

3

7163

0

7169

Other

79

0

0

1

2273672

2573752

Total

77651

5224

1427

7169

2573747

2665218

After discussion at C&A meeting I tried to make a few more high-level tests, comparing events in gamma-ray classes (in either lp60 or P202) and the fraction of events with difference in energy or angle larger than some threshold.

Direction P202-lp60 angle:

Class

Tot

log(Angle)>-3.40 (~0.023 deg)

Transient

91546

259 (0.3 %)

Source

13826

13 (0.1 %)

Clean

8600

5 (0.1 %)

UltraClean

7175

4 (0.1 %)

Plot:

 

Energy P202-lp60 ratio:

Class

Tot

|L1.E/E-1|>1.0%

|L1.E/E-1|>5.0%

|L1.E/E-1|>10.0%

Transient

91546

257 (0.3 %)

124 (0.1 %)

76 (0.001 %)

Source

13826

56 (0.4 %)

28 (0.2 %)

12 (0.001 %)

Clean

8600

39 (0.5 %)

23 (0.3 %)

10 (0.0012 %)

UltraClean

7175

38 (0.5 %)

23 (0.3 %)

10 (0.0014 %)

Plot:

 

Variations seem quite small. Large change in Energy are due change in Energy Method

Another comparison between lp60 and P202

Here is the log10(angular separation) plot for lp60:

As Carmelo just posted, the improvement for the tkr variables is fantastic!

MeritTuple test of run 388740704 (Carmelo)

A second run was processed in L1Proc with GR 17-35-24-lp60 in order to have more statistics and do some high-level study with some bright source. Run 388740704 has been chosen at it contains GRB130427A with several high energy photons.

Unfortunately, event if the calibration flavor settings is correct, we used in L1proc test different calibration constants. You can scan the log file for details, here an example with CAL_Ped:
the same query "SELECT ser_no FROM metadata_v2r1 WHERE ((completion="OK") AND (instrument="LAT") AND (calib_type="CAL_Ped") AND (flavor="p7repro") AND ("2013-04-27 07:31:47">vstart) AND ("2013-04-27 07:31:47"<vend) AND ("PROD"=PROC_LEVEL)) ORDER BY update_time desc" returned file "$(LATCalibRoot)/CAL/p7repro/pedavr_382m_386m.xml" in P202 and file "$(LATCalibRoot)/CAL/p7repro/pedavr_388m_392m.xml" in this test. As a consequence, this test is not a comparison of GR lp60 vs gr17 as in previous run, but a real end-to-end test of how this run would look like if we had to reprocess it now (at least for the CAL calib). In other words, it includes the differences in GR and small differences in CAL calibrations.

Here the tables and plots of the fraction of events with difference in energy or angle larger than some threshold. As expected there are many more events now with some difference, the small update in in calibration seem to be the dominant effect.

Direction P202-lp60 angle:

Class

Tot

log(Angle)>-3.40 (~0.023 deg)

Transient

215400

20829 (9.7 %)

Source

63401

1797 (2.8 %)

Clean

49960

1351 (2.7 %)

UltraClean

41597

1110 (2.7 %)

Plot:

 

Energy P202-lp60 ratio:

Class

Tot

|L1.E/E-1|>5.0%

Transient

215400

1069 (0.5 %)

Source

63401

197 (0.3 %)

Clean

49960

157 (0.3 %)

UltraClean

41597

126 (0.3 %)

Plot:

 

Another check is on the high-energy photons found in GRB130427A analysis. I scanned the same events as I did for the GRB see here for details, and observed essentially no differences for these events:

t.Scan("EvtElapsedTime-388741629:CTBBestEnergy:L1.CTBBestEnergy:abs(L1.CTBBestEnergy/CTBBestEnergy-1):(FT1EventClass&0x1d):FT1EventClass==L1.FT1EventClass): CTBCORE:L1.CTBCORE:CTBBestEnergyProb:L1.CTBBestEnergyProb:FT1Ra:L1.FT1Ra:FT1Dec:L1.FT1Dec",
"(EvtEventId==3390429) || (EvtEventId==2464261) || (EvtEventId==3444739) || (EvtEventId==4705506) || (EvtEventId==11983827) || (EvtEventId==2580344) || (EvtEventId==2731009) || (EvtEventId==2963451) || (EvtEventId==3283498) || (EvtEventId==2713633) || (EvtEventId==3268741) || (EvtEventId==2648342) || (EvtEventId==2482436)")

************************************************************************************************************************************************************************************
*    Row   * EvtElapse * CTBBestEn * L1.CTBBes * abs(L1.CT * (FT1Event * (FT1Event *   CTBCORE * L1.CTBCOR * CTBBestEn * L1.CTBBes *     FT1Ra *  L1.FT1Ra *    FT1Dec * L1.FT1Dec *
************************************************************************************************************************************************************************************
*   472948 * 19.064351 * 71305.953 * 71213.570 * 0.0012955 *         0 *         1 *         0 *         0 * 0.6706129 * 0.6701418 * 171.99769 * 171.99771 * 31.862634 * 31.862632 *
*   476376 * 23.645536 * 10775.267 * 10780.360 * 0.0004726 *        29 *         1 * 0.0445221 * 0.0445221 * 0.8618040 * 0.8618040 * 172.17590 * 172.15327 * 27.288623 * 27.295263 *
*   494903 * 48.007806 * 28708.009 * 28603.675 * 0.0036343 *         5 *         1 * 0.6221749 * 0.5748100 * 0.2407225 * 0.2446980 * 173.28053 *  173.2827 * 27.651189 * 27.650981 *
*   507524 * 64.894754 * 11988.679 * 11930.457 * 0.0048564 *        29 *         1 * 0.8034922 * 0.8034922 * 0.3546319 * 0.3373169 * 173.31433 * 173.31712 * 27.725812 * 27.723766 *
*   519623 * 80.940313 * 13035.263 * 13009.251 * 0.0019955 *         5 *         1 * 0.8435378 * 0.8435378 * 0.2496242 * 0.2447519 * 173.25058 * 173.24089 * 27.677135 * 27.680463 *
*   522826 * 85.159673 * 27310.296 * 27199.644 * 0.0040516 *         5 *         1 * 0.1386326 * 0.0806780 * 0.2626658 * 0.2752659 * 173.39186 * 173.46028 * 27.820808 * 27.834432 *
*   566726 * 141.52865 * 25199.880 * 25200.130 * 9.920e-06 *         5 *         1 * 0.6293173 * 0.5874550 * 0.7047906 * 0.7251923 * 173.22123 * 173.22604 * 27.811012 * 27.812912 *
*   627305 * 214.35738 *  11977.75 * 11974.966 * 0.0002323 *        29 *         1 * 0.8143669 * 0.8454899 * 0.8455054 * 0.8470528 * 172.80119 * 172.80377 * 27.971977 * 27.975347 *
*   630337 * 217.88755 * 15591.344 * 15570.979 * 0.0013061 *        29 *         1 * 0.5907437 * 0.6044979 * 0.6589291 * 0.6541520 * 173.06861 * 173.06947 * 27.659246 * 27.660011 *
*   652206 * 243.54966 * 98431.859 * 98157.437 * 0.0027879 *        29 *         1 * 0.6527069 * 0.6527069 * 0.7233405 * 0.7233405 * 173.09648 * 173.09504 * 27.704078 * 27.704475 *
*   663476 * 256.70178 * 49680.613 * 49719.105 * 0.0007747 *         5 *         1 * 0.7985399 * 0.8340377 * 0.7023981 * 0.6979178 * 173.19381 * 173.19599 * 27.637081 * 27.639444 *
*   937454 * 611.00936 * 43991.320 * 44033.296 * 0.0009542 *        29 *         1 * 0.9545454 * 0.9890909 * 0.7674124 * 0.7106212 * 173.17210 * 173.17063 * 27.676036 * 27.674320 *
*  2552075 * 3410.2581 * 39623.308 * 39602.593 * 0.0005227 *        29 *         1 * 0.8786774 * 0.9299595 * 0.7597252 * 0.7537148 * 173.13639 * 173.13305 * 27.720428 * 27.718521 *
************************************************************************************************************************************************************************************
==> 13 selected entries

Event class table:

L1\P202

Transient

Source

Clean

UltraClean

Other

Total

Transient

149982

137

153

97

990

151359

Source

123

13162

2

15

14

13316

Clean

152

4

8031

243

15

8445

UltraClean

82

14

216

40892

26

41230

Other

1027

5

6

12

3108814

3109864

Total

151366

13322

8408

41259

3109859

3324214

Comparison of AllGamma simulation

AG-GR-v17r35p24lp60-SL6 done in Lyon (JIRA SIMREQ-134 , DataCatalog link ) compared with AG-GR-v17r35p24lp43-allE-2 used for IRF P7Rep V15.
I checked the faction of events that survive the Transient, Source and Clean cut (plots below). The differences are very small (~5% at 100 MeV). I think it is safe to use this GR in L1Proc.

Philippe made a direct comparison between data and MC efficiencies (from Transient to Clean), observing same behavior between lp60 and lp43:

Reason for change

This is the version needed for P7REP. The switchover is planned for Tuesday, November 5th.

Test Procedure

We have processed several LPA and LCI runs in the DEV pipeline with this version of L1Proc.

We fully reprocessed a couple of runs, which are fully described in the Validations Page.

Rollback procedure

We can easily switch back to the previous version of L1Proc before the FSSC switches over to the new photon database (we probably have a few hours to rollback, if needed). This would have to be coordinated with ASP.

CCB Jira

SSC-358@JIRA

Details

L1Pipeline: L1Pipeline-04-04-00
- Do not create SVAC tuples
- Change procver to 202 for all exported file types
- New GR GlastRelease-17-35-24-lp61
- New ST ScienceTools-09-32-05
- New DiffRsp, new ft2Utils, new fitsGen for P202
- Add ability to ignore some deliveries when merging LONE-190@JIRA
- Don't clean up chunks until we're really done LONE-193@JIRA
- Improve retry handling LONE-194@JIRA

ScienceTools: ScienceTools-09-32-05

GlastRelease: GlastRelease-17-35-24-lp61
- There are several differences between GlastRelease-17-35-24-lpr61 and GlastRelease-17-35-24-lp40 (currently used in L1Proc). Most packages were updated to keep up with the Science Tools release.
- We ran Systests comparing GlastRelease-17-35-24-lp61 and GlastRelease-17-35-24-lp40 and there are several differences in Overlay and BackGndMixDC2. We think we understand these differences in terms of the updated flux & celestialSources in the updated version of astro (plus a bugfix introduced in Overlay starting from GlastRelease-17-35-24-lp43).
- We performed extensive validations of this Glast Release and of the associated calibration flavor, L1Current. They are described in the Validations Page.

evtClassDefs: evtClassDefs-00-19-05
- add for PASS_VER = P7REP

fitsGen: fitsGen-06-06-05
- ensure MeritFile2::next() does not try extend beyond tuple size
- call facilities::commonUtilities::setupEnvironment() (again and again) so that getPackagePath doesn't crash because of a null pointer
- remove st_stream dependence
- check TTree->GetEvent return code
- patch tag since ft2.tpl was missed somehow by the fitsGen-06-06-00 tag
- add LAMBDA to ft2.tpl
- for Windows no-install-headers
- add PROC_VER keyword to ft2.tpl PHDU
- add #include <vector> to MeritFile2.h for gcc4.4.3/Ubuntu
- bug-fix in ~MeritFile2 for handling m_tree when using it as a TChain
- add constructor to MeritFile2 that takes a list of input merit files

ft2Util: ft2Util-02-02-00
- Fixed a problem with static initialization which was causing seg. fault because of the changed behavior of facilities::commonUtilities::getDataPath
- Adding Lambda to the FT2 file. This has to be linked with a GR containing astro > astro-03-13-03 and fitsGen > fitsGen-06-06-00. The new version of astro contains also a new Earth magnetic model.

Complete set of tags for L1Proc 4.4

GlastRelease: GlastRelease-17-35-24-lp61

ScienceTools: ScienceTools-09-32-05

svac/L1Pipeline: L1Pipeline-04-04-00

calibGenTKR: calibGenTKR-04-08-01
calibTkrUtil: calibTkrUtil-02-09-06-gr02
fitsGen: fitsGen-06-06-05

dataMonitoring/AlarmsCfg: AlarmsCfg-06-01-01
dataMonitoring/Common: Common-06-11-05
dataMonitoring/DigiReconCalMeritCfg: DigiReconCalMeritCfg-01-20-02
dataMonitoring/FastMon: FastMon-05-03-00
dataMonitoring/FastMonCfg: FastMonCfg-02-02-01
datMonitoring/IGRF: IGRF-02-01-00

svac/Monitor: Monitor-02-01-04
svac/EngineeringModelRoot: EngineeringModelRoot-05-00-00
svac/TestReport: TestReport-12-01-00
svac/findGaps: findGaps-02-02-00

users/richard/pipelineDatasets: pipelineDatasets-01-00-00
ft2Util: ft2Util-02-02-00

evtClassDefs: evtClassDefs-00-19-05
GPLtools: GPLtools-02-00-00-wf02

ROOT: ROOT v5.26.00a-gl6