We are reprocessing run 351769427 several times, with different settings.
L1Proc 3.0-PROD, GR 17-35-24-gr15, ST 09-24-00, calib flavor vanilla
L1Proc stream: http://glast-ground.slac.stanford.edu/Pipeline-II/exp/Fermi/si.jsp?stream=76321789&dataSourceMode=Prod
First log file (digi): /nfs/farm/g/glast/u41/L1/logs/PROD/L1Proc/3.0/doRun/doChunk/digitization/120xxxxxx/225xxx/001/351xxxxxx/769xxx/427/096xxx/180/logFile.txt
Digi job options (in order of inclusion, NOT in order of execution):
- Digitization: /afs/slac.stanford.edu/g/glast/ground/releases/volume14/GlastRelease-17-35-24-gr15/jobOptions/Gleam/jobOptions/pipeline/ldf2digi.txt
- Calibrations: /afs/slac.stanford.edu/g/glast/ground/releases/volume14/GlastRelease-17-35-24-gr15/jobOptions/Gleam/jobOptions/pipeline/calibrations.txt
- Geometry: /afs/slac.stanford.edu/g/glast/ground/releases/volume14/GlastRelease-17-35-24-gr15/xml/xmlGeoDbs/flight/flightSegVols.xml
- Pipeline: /afs/slac.stanford.edu/g/glast/ground/PipelineConfig/Level1/3.0/data/digi.jobOpt
First log file (recon): /nfs/farm/g/glast/u41/L1/logs/PROD/L1Proc/3.0/doRun/doChunk/doCrumb/recon/120xxxxxx/225xxx/001/351xxxxxx/769xxx/427/096xxx/180/000/logFile.txt
Recon job options (in order of inclusion, NOT in order of execution):
- Reconstruction: /afs/slac.stanford.edu/g/glast/ground/releases/volume14/GlastRelease-17-35-24-gr15/jobOptions/Gleam/jobOptions/pipeline/readigi_runrecon.txt
- Calibrations: /afs/slac.stanford.edu/g/glast/ground/releases/volume14/GlastRelease-17-35-24-gr15/jobOptions/Gleam/jobOptions/pipeline/calibrations.txt
- Geometry: /afs/slac.stanford.edu/g/glast/ground/releases/volume14/GlastRelease-17-35-24-gr15/xml/xmlGeoDbs/flight/flightSegVols.xml
- Pipeline: /afs/slac.stanford.edu/g/glast/ground/PipelineConfig/Level1/3.0/data/recon.jobOpt
Output files:
- Digi: root://glast-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/prod/3.0/digi/r0351769427_v000_digi.root
- Merit: root://glast-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/prod/3.0/merit/r0351769427_v000_merit.root
- Recon: root://glast-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/prod/3.0/recon/r0351769427_v000_recon.root
- FT1: root://glast-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/prod/3.0/ft1/gll_ph_p120_r0351769427_v000.fit
L1Proc_a 4.6-DEV, GR 17-35-24-lp60, ST 09-32-05, calib flavor L1current
L1Proc stream: http://glast-ground.slac.stanford.edu/Pipeline-II/exp/Fermi/si.jsp?stream=2318938&dataSourceMode=Dev
First log file (digi): /nfs/farm/g/glast/u41/L1/logs/DEV/L1Proc_a/4.6/doRun/doChunk/digitization/120xxxxxx/225xxx/001/351xxxxxx/769xxx/427/096xxx/180/logFile.txt
Digi job options (in order of inclusion, NOT in order of execution):
- Digitization: /afs/slac.stanford.edu/g/glast/ground/releases/volume10/GlastRelease-17-35-24-lp60/jobOptions/Gleam/jobOptions/pipeline/ldf2digi_a.txt
- Calibrations: /afs/slac.stanford.edu/g/glast/ground/releases/volume10/GlastRelease-17-35-24-lp60/jobOptions/Gleam/jobOptions/pipeline/calibrations_a.txt
- Geometry: /afs/slac.stanford.edu/g/glast/ground/releases/volume10/GlastRelease-17-35-24-lp60/xml/xmlGeoDbs/flight01/flightSegVols.xml
- Pipeline: /afs/slac.stanford.edu/g/glast/ground/PipelineConfig/Level1/4.6/data/digi.jobOpt_a
First log file (recon): /nfs/farm/g/glast/u41/L1/logs/DEV/L1Proc_a/4.6/doRun/doChunk/doCrumb/recon/120xxxxxx/225xxx/001/351xxxxxx/769xxx/427/096xxx/180/000/logFile.txt
Recon job options (in order of inclusion, NOT in order of execution):
- Reconstruction: /afs/slac.stanford.edu/g/glast/ground/releases/volume10/GlastRelease-17-35-24-lp60/jobOptions/Gleam/jobOptions/pipeline/readigi_runrecon_a.txt
- Calibrations: /afs/slac.stanford.edu/g/glast/ground/releases/volume10/GlastRelease-17-35-24-lp60/jobOptions/Gleam/jobOptions/pipeline/calibrations_a.txt
- Geometry: /afs/slac.stanford.edu/g/glast/ground/releases/volume10/GlastRelease-17-35-24-lp60/xml/xmlGeoDbs/flight01/flightSegVols.xml
- Pipeline: /afs/slac.stanford.edu/g/glast/ground/PipelineConfig/Level1/4.6/data/recon.jobOpt_a
Output files:
- Digi: root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/4.6/digi/r0351769427_v020_digi.root
- Merit: root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/4.6/merit/r0351769427_v020_merit.root
- Recon: root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/4.6/recon/r0351769427_v016_recon.root
- FT1: root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/4.6/ft1/gll_ph_p202_r0351769427_v019.fit
L1Proc_b 4.6-DEV, run 388740704, GR 17-35-24-lp60, ST 09-32-05, calib flavor (selected overrides): p7repro
L1Proc stream: http://glast-ground.slac.stanford.edu/Pipeline-II/exp/Fermi/si.jsp?stream=2322147&dataSourceMode=Dev
First log file (digi): /nfs/farm/g/glast/u41/L1/logs/DEV/L1Proc_b/4.6/doRun/doChunk/digitization/388xxxxxx/740xxx/704/388xxxxxx/740xxx/704/116xxx/232/logFile.txt
Digi job options (in order of inclusion, NOT in order of execution):
- Digitization: /afs/slac.stanford.edu/g/glast/ground/releases/volume10/GlastRelease-17-35-24-lp60/jobOptions/Gleam/jobOptions/pipeline/ldf2digi_b.txt
- Calibrations: /afs/slac.stanford.edu/g/glast/ground/releases/volume10/GlastRelease-17-35-24-lp60/jobOptions/Gleam/jobOptions/pipeline/calibrations_b.txt
- Geometry: /afs/slac.stanford.edu/g/glast/ground/releases/volume10/GlastRelease-17-35-24-lp60/xml/xmlGeoDbs/flight/flightSegVols.xml
- Pipeline: /afs/slac.stanford.edu/g/glast/ground/PipelineConfig/Level1/4.6/data/digi.jobOpt_b
First log file (recon): /nfs/farm/g/glast/u41/L1/logs/DEV/L1Proc_b/4.6/doRun/doChunk/doCrumb/recon/388xxxxxx/740xxx/704/388xxxxxx/740xxx/704/116xxx/232/000/logFile.txt
Recon job options (in order of inclusion, NOT in order of execution):
- Reconstruction: /afs/slac.stanford.edu/g/glast/ground/releases/volume10/GlastRelease-17-35-24-lp60/jobOptions/Gleam/jobOptions/pipeline/readigi_runrecon_b.txt
- Calibrations: /afs/slac.stanford.edu/g/glast/ground/releases/volume10/GlastRelease-17-35-24-lp60/jobOptions/Gleam/jobOptions/pipeline/calibrations_b.txt
- Geometry: /afs/slac.stanford.edu/g/glast/ground/releases/volume10/GlastRelease-17-35-24-lp60/xml/xmlGeoDbs/flight01/flightSegVols.xml
- Pipeline: /afs/slac.stanford.edu/g/glast/ground/PipelineConfig/Level1/4.6/data/recon.jobOpt_b
Output files:
- Digi: root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/4.6/digi/r0388740704_v000_digi.root
- Merit: root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/4.6/merit/r0388740704_v000_merit.root
- Recon: root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/4.6/recon/r0388740704_v000_recon.root
- FT1: root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/4.6/ft1/gll_ph_p202_r0388740704_v000.fit
L1Proc_a 4.4-DEV, GR 17-35-24-lp57, ST 09-32-05, calib flavor L1current
L1Proc stream: http://glast-ground.slac.stanford.edu/Pipeline-II/exp/Fermi/si.jsp?stream=2293138&dataSourceMode=Dev
First log file (digi): /nfs/farm/g/glast/u41/L1/logs/DEV/L1Proc_a/4.4/doRun/doChunk/digitization/120xxxxxx/225xxx/001/351xxxxxx/769xxx/427/096xxx/180/logFile.txt
Digi job options (in order of inclusion, NOT in order of execution):
- Digitization: /afs/slac.stanford.edu/g/glast/ground/releases/volume10/GlastRelease-17-35-24-lp57/jobOptions/Gleam/jobOptions/pipeline/ldf2digi_a.txt
- Calibrations: /afs/slac.stanford.edu/g/glast/ground/releases/volume10/GlastRelease-17-35-24-lp57/jobOptions/Gleam/jobOptions/pipeline/calibrations_a.txt
- Geometry: /afs/slac.stanford.edu/g/glast/ground/releases/volume10/GlastRelease-17-35-24-lp57/xml/xmlGeoDbs/flight01/flightSegVols.xml
- Pipeline: /afs/slac.stanford.edu/g/glast/ground/PipelineConfig/Level1/4.4/data/digi.jobOpt_a
First log file (recon): /nfs/farm/g/glast/u41/L1/logs/DEV/L1Proc_a/4.4/doRun/doChunk/doCrumb/recon/120xxxxxx/225xxx/001/351xxxxxx/769xxx/427/096xxx/180/000/logFile.txt
Recon job options (in order of inclusion, NOT in order of execution):
- Reconstruction: /afs/slac.stanford.edu/g/glast/ground/releases/volume10/GlastRelease-17-35-24-lp57/jobOptions/Gleam/jobOptions/pipeline/readigi_runrecon_a.txt
- Calibrations: /afs/slac.stanford.edu/g/glast/ground/releases/volume10/GlastRelease-17-35-24-lp57/jobOptions/Gleam/jobOptions/pipeline/calibrations_a.txt
- Geometry: /afs/slac.stanford.edu/g/glast/ground/releases/volume10/GlastRelease-17-35-24-lp57/xml/xmlGeoDbs/flight01/flightSegVols.xml
- Pipeline: /afs/slac.stanford.edu/g/glast/ground/PipelineConfig/Level1/4.4/data/recon.jobOpt_a
Output files:
- Digi: root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/4.4/digi/r0351769427_v014_digi.root
- Merit: root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/4.4/merit/r0351769427_v014_merit.root
- Recon: root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/4.4/recon/r0351769427_v011_recon.root
- FT1: root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/4.4/ft1/gll_ph_p202_r0351769427_v014.fit
L1Proc_b 4.4-DEV, GR 17-35-24-lp57, ST 09-32-05, calib flavor (selected overrides): p7repro
L1Proc stream: http://glast-ground.slac.stanford.edu/Pipeline-II/exp/Fermi/si.jsp?stream=2296311&dataSourceMode=Dev
First log file (digi): /nfs/farm/g/glast/u41/L1/logs/DEV/L1Proc_b/4.4/doRun/doChunk/digitization/120xxxxxx/225xxx/001/351xxxxxx/769xxx/427/096xxx/180/logFile.txt
Digi job options (in order of inclusion, NOT in order of execution):
- Digitization: /afs/slac.stanford.edu/g/glast/ground/releases/volume10/GlastRelease-17-35-24-lp57/jobOptions/Gleam/jobOptions/pipeline/ldf2digi_b.txt
- Calibrations: /afs/slac.stanford.edu/g/glast/ground/releases/volume10/GlastRelease-17-35-24-lp57/jobOptions/Gleam/jobOptions/pipeline/calibrations_b.txt
- Geometry: /afs/slac.stanford.edu/g/glast/ground/releases/volume10/GlastRelease-17-35-24-lp57/xml/xmlGeoDbs/flight/flightSegVols.xml
- Pipeline: /afs/slac.stanford.edu/g/glast/ground/PipelineConfig/Level1/4.4/data/digi.jobOpt_b
First log file (recon): /nfs/farm/g/glast/u41/L1/logs/DEV/L1Proc_b/4.4/doRun/doChunk/doCrumb/recon/120xxxxxx/225xxx/001/351xxxxxx/769xxx/427/096xxx/180/000/logFile.txt
Recon job options (in order of inclusion, NOT in order of execution):
- Reconstruction: /afs/slac.stanford.edu/g/glast/ground/releases/volume10/GlastRelease-17-35-24-lp57/jobOptions/Gleam/jobOptions/pipeline/readigi_runrecon_b.txt
- Calibrations: /afs/slac.stanford.edu/g/glast/ground/releases/volume10/GlastRelease-17-35-24-lp57/jobOptions/Gleam/jobOptions/pipeline/calibrations_b.txt
- Geometry: /afs/slac.stanford.edu/g/glast/ground/releases/volume10/GlastRelease-17-35-24-lp57/xml/xmlGeoDbs/flight01/flightSegVols.xml
- Pipeline: /afs/slac.stanford.edu/g/glast/ground/PipelineConfig/Level1/4.4/data/recon.jobOpt_b
Output files:
- Digi: root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/4.4/digi/r0351769427_v015_digi.root
- Merit: root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/4.4/merit/r0351769427_v015_merit.root
- Recon: root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/4.4/recon/r0351769427_v012_recon.root
- FT1: root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/4.4/ft1/gll_ph_p202_r0351769427_v015.fit
L1Proc_a 4.5-DEV, GR 17-35-24-gr35*, ST 09-32-05, calib flavor L1current
L1Proc stream: http://glast-ground.slac.stanford.edu/Pipeline-II/exp/Fermi/si.jsp?stream=2306040&dataSourceMode=Dev
First log file (digi): /nfs/farm/g/glast/u41/L1/logs/DEV/L1Proc_a/4.5/doRun/doChunk/digitization/120xxxxxx/225xxx/001/351xxxxxx/769xxx/427/096xxx/180/logFile.txt
Digi job options (in order of inclusion, NOT in order of execution):
- Digitization: /afs/slac.stanford.edu/g/glast/ground/releases/volume07/GlastRelease-17-35-24-gr35/jobOptions/Gleam/jobOptions/pipeline/ldf2digi_a.txt
- Calibrations: /afs/slac.stanford.edu/g/glast/ground/releases/volume07/GlastRelease-17-35-24-gr35/jobOptions/Gleam/jobOptions/pipeline/calibrations_a.txt
- Geometry: /afs/slac.stanford.edu/g/glast/ground/releases/volume07/GlastRelease-17-35-24-gr35/xml/xmlGeoDbs/flight01/flightSegVols.xml
- Pipeline: /afs/slac.stanford.edu/g/glast/ground/PipelineConfig/Level1/4.5/data/digi.jobOpt_a
First log file (recon): /nfs/farm/g/glast/u41/L1/logs/DEV/L1Proc_a/4.5/doRun/doChunk/doCrumb/recon/120xxxxxx/225xxx/001/351xxxxxx/769xxx/427/096xxx/180/000/logFile.txt
Recon job options (in order of inclusion, NOT in order of execution):
- Reconstruction: /afs/slac.stanford.edu/g/glast/ground/releases/volume07/GlastRelease-17-35-24-gr35/jobOptions/Gleam/jobOptions/pipeline/readigi_runrecon_a.txt
- Calibrations: /afs/slac.stanford.edu/g/glast/ground/releases/volume07/GlastRelease-17-35-24-gr35/jobOptions/Gleam/jobOptions/pipeline/calibrations_a.txt
- Geometry: /afs/slac.stanford.edu/g/glast/ground/releases/volume07/GlastRelease-17-35-24-gr35/xml/xmlGeoDbs/flight01/flightSegVols.xml
- Pipeline: /afs/slac.stanford.edu/g/glast/ground/PipelineConfig/Level1/4.5/data/recon.jobOpt_a
Output files:
- Digi: root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/4.5/digi/r0351769427_v018_digi.root
- Merit: root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/4.5/merit/r0351769427_v018_merit.root
- Recon: root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/4.5/recon/r0351769427_v014_recon.root
- FT1: root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/4.5/ft1/gll_ph_p202_r0351769427_v017.fit
* Note: 17-35-24-gr35 is the oldest GR which can be compiled with rhel5 and therefore tested in the current L1Proc setup.
L1Proc_b 4.5-DEV, GR 17-35-24-gr35*, ST 09-32-05, calib flavor (selected overrides): p7repro
L1Proc stream: http://glast-ground.slac.stanford.edu/Pipeline-II/exp/Fermi/si.jsp?stream=2315944&dataSourceMode=Dev
First log file (digi): /nfs/farm/g/glast/u41/L1/logs/DEV/L1Proc_b/4.5/doRun/doChunk/digitization/120xxxxxx/225xxx/001/351xxxxxx/769xxx/427/096xxx/180/logFile.txt
Digi job options (in order of inclusion, NOT in order of execution):
- Digitization: /afs/slac.stanford.edu/g/glast/ground/releases/volume07/GlastRelease-17-35-24-gr35/jobOptions/Gleam/jobOptions/pipeline/ldf2digi_b.txt
- Calibrations: /afs/slac.stanford.edu/g/glast/ground/releases/volume07/GlastRelease-17-35-24-gr35/jobOptions/Gleam/jobOptions/pipeline/calibrations_b.txt
- Geometry: /afs/slac.stanford.edu/g/glast/ground/releases/volume07/GlastRelease-17-35-24-gr35/xml/xmlGeoDbs/flight/flightSegVols.xml
- Pipeline: /afs/slac.stanford.edu/g/glast/ground/PipelineConfig/Level1/4.5/data/digi.jobOpt_b
First log file (recon): /nfs/farm/g/glast/u41/L1/logs/DEV/L1Proc_b/4.5/doRun/doChunk/doCrumb/recon/120xxxxxx/225xxx/001/351xxxxxx/769xxx/427/096xxx/180/000/logFile.txt
Recon job options (in order of inclusion, NOT in order of execution):
- Reconstruction: /afs/slac.stanford.edu/g/glast/ground/releases/volume07/GlastRelease-17-35-24-gr35/jobOptions/Gleam/jobOptions/pipeline/readigi_runrecon_b.txt
- Calibrations: /afs/slac.stanford.edu/g/glast/ground/releases/volume07/GlastRelease-17-35-24-gr35/jobOptions/Gleam/jobOptions/pipeline/calibrations_b.txt
- Geometry: /afs/slac.stanford.edu/g/glast/ground/releases/volume07/GlastRelease-17-35-24-gr35/xml/xmlGeoDbs/flight01/flightSegVols.xml
- Pipeline: /afs/slac.stanford.edu/g/glast/ground/PipelineConfig/Level1/4.5/data/recon.jobOpt_b
Output files:
- Digi: root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/4.5/digi/r0351769427_v019_digi.root
- Merit: root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/4.5/merit/r0351769427_v019_merit.root
- Recon: root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/4.5/recon/r0351769427_v015_recon.root
- FT1: root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/4.5/ft1/gll_ph_p202_r0351769427_v018.fit
* Note: 17-35-24-gr35 is the oldest GR which can be compiled with rhel5 and therefore tested in the current L1Proc setup.
P202-ROOT, P7REP reprocessing pipeline task, GR 17-35-24-gr17, calibration flavor (selected overrides): p7repro
- Total events: 2665218
- Date reprocessed: 25 May 2012 (ROOT files), 12 June 2013 (FITS files)
- Number of 'clumps' (number of different CPU cores used by Gleam in P202-ROOT): 27
- Each clump processed approx 98710 events
- Top-level pipeline stream (P202-ROOT & P202-FITS): 19672
- Location of the 27 P202-ROOT log files: /nfs/farm/g/glast/u38/Reprocess-tasks/P202-ROOT/output/P202-ROOT/1.0/processRun/processClump/019xxx/672/000/*/logFile.txt
- Web-accessible path to logs:http://glast-ground.slac.stanford.edu/Decorator/exp/Fermi/Decorate/u38/Reprocess-tasks/P202-ROOT/output/P202-ROOT/1.0/processRun/processClump/019xxx/672
- JobOptions file:
- generic for P202: http://glast-ground.slac.stanford.edu/Decorator/exp/Fermi/Decorate/u38/Reprocess-tasks/P202-ROOT/config/doRecon.txt
- specific for clump-0 of this run: http://glast-ground.slac.stanford.edu/Decorator/exp/Fermi/Decorate/u38/Reprocess-tasks/P202-ROOT/output/P202-ROOT/1.0/processRun/processClump/019xxx/672/000/doRecon.txt
- More details about the configuration of this task: https://confluence.slac.stanford.edu/display/~dragon/P202+-+Full+Pass7+Reprocessing
- Input file:
- DIGI: root://glast-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/prod/3.0/digi/r0351769427_v000_digi.root
- Output files (selected):
- MERIT: root://glast-rdr.slac.stanford.edu//glast/Data/Flight/Reprocess/P202/merit/r0351769427_v202_merit.root
- FT1: root://glast-rdr.slac.stanford.edu//glast/Data/Flight/Reprocess/P202/ft1/gll_ph_p202_r0351769427_v203.fit
- EXTENDEDFT1: root://glast-rdr.slac.stanford.edu//glast/Data/Flight/Reprocess/P202/extendedft1/gll_xp_p202_r0351769427_v203.fit
- RECON: root://glast-rdr.slac.stanford.edu//glast/Data/Flight/Reprocess/P202/recon/r0351769427_v202_recon.root
- Additional detail
P202-ROOT, run 388740704, GR 17-35-24-gr17, calibration flavor (selected overrides): p7repro
- Example log file: /nfs/farm/g/glast/u38/Reprocess-tasks/P202-ROOT/output/P202-ROOT/1.0/processRun/processClump/026xxx/201/000/logFile.txt
- Web link: http://glast-ground.slac.stanford.edu/Decorator/exp/Fermi/Decorate/u38/Reprocess-tasks/P202-ROOT/output/P202-ROOT/1.0/processRun/processClump/026xxx/201/000/
- Output files:
- MERIT: root://glast-rdr.slac.stanford.edu//glast/Data/Flight/Reprocess/P202/merit/r0388740704_v202_merit.root
Summary of the flavors for P202 and the new L1
P202
All calibs are set to vanilla, except for the following eight, which are set to p7repro:
TKR_HotChan, _DeadChan, _TowerAlign, _InternalAlign
CAL_Ped, _MevPerDac, _Asym
ACD_Ped, _ElecGain, _HighRange, _CoherentNoise
NAS_LATAlignment
The corresponding JO:
TkrReconCalib.calibFlavor = "vanilla";
TkrReconCalib.hotStripsCalibFlavor = "p7repro";
TkrReconCalib.deadStripsCalibFlavor = "p7repro";
TkrReconCalib.towerAlignmentCalibFlavor = "p7repro";
TkrReconCalib.internalAlignmentCalibFlavor = "p7repro";
Note: in some tasks that start with digitization, "TkrReconCalib" will be called "TkrCalibAlg".
CalCalibSvc.DefaultFlavor = "vanilla";
CalCalibSvc.FlavorPed = "p7repro";
CalCalibSvc.FlavorMevPerDac = "p7repro";
CalCalibSvc.FlavorAsym = "p7repro";
AcdCalibSvc.DefaultFlavor = "vanilla";
AcdCalibSvc.Ped = "p7repro";
AcdCalibSvc.Gain = "p7repro";
AcdCalibSvc.HighRange = "p7repro";
AcdCalibSvc.FlavorCoherentNoise = "p7repro";
New L1
All calibrations use flavor L1current, and the JO is:
TkrReconCalib.calibFlavor = "L1current";
CalCalibSvc.DefaultFlavor = "L1current";
AcdCalibSvc.DefaultFlavor = "L1current";
Comparison of merit files (Philippe)
Here are some comparisons between the two merit files (confirming Carmelo's initial findings):
The nb of tracks can be very different. When looking at the ang separation between the tkr dir of the two files or the cal dir of the two files, it can be very large (when log10(ang sep)<-3, it is due to round off). The difference in tkr direction is not correlated to the difference in cal direction.
Test with MC (Philippe)
I've produced some MC (AG w/o overlay with 2.5<logE<3.5 and -1.0<ZDir<-0.5) with lp40 and lp56, using exactly the same job options and calibrations.
Here are the comparison plots:
There is no difference (within round off). So the issue we have comes from either calibration or job options.
Comparison of merit files (Carmelo)
Just a few more info, for my first analysis:
Cal centroid differences in X and Y are larger when xtals are all in the same layer, in particular the top and the bottom layer (no difference on Z, only 7 events have different CalZEcntr):
Looking at high-level variables like CalTransRms: 63013 evts differ (~2%), mostly in case of very low CalTransRms ( e.g.56269 have log10(CalTransRms)<-2 ).
TkrNumTracks: 78547 evts differ. Here a scatter plot of these events (Z axis in log scale), mostly +-1 track (still strange), but sometime there are huge differences.
Calibration Comparison (Joanne)
I compared calibration-related output from P202-ROOT (the excerpt Tom included in his email) and the sample 4.4 log file M.E. suggested above. The collection of calibrations (comparing filepaths) is almost identical, however the order in which they are fetched is not, which could indicate the processing steps using those calibrations are not occurring in the same order either, or (looking more likely) that the files I compared were not processing the same data. Different events can cause calibrations to be fetched in a different order.
I checked the dates on all calibration files used. They are all as old as they should be; that is, there is no indication that any of them have been modified since they were first put into use.
The one difference is in splits. For p202 the file used is $(LATCalibRoot)/TKR/TkrSplits_default.xml
For 4.4 it's $(LATCalibRoot)/TKR/TkrSplits_flight.xml
(Carmelo) The difference between default and flight is that in the latter bottom layer in tower 10 (x=2, y=2) is read out from one end only. Not sure how this info is used in reconstruction, but 'Hit' variables in MeritTuple (like TkrNumHits, TkrHitsInLyrNN, etc ) are identical in the 2 cases. The problems comes only when recon is involved.
Comparison of recon files (Sasha)
I've printed the crystal energies and positions for 11 events with big difference in energy centroid position between pass7rep (attached) and level1new (attached), and I can confirm that energies and positions reconstructed in individual crystals are identical for these two files, while X and Y centroid positions have dramatic difference.
(Carmelo) Thanks Sasha! This means that the problem is not in calibration or rounding, we are somehow using a different algorithm.
Notice that the values in centroid calculation correspond to the energy-weighted average using all xtals in level1new and only some of the xtals in P202.
Comparison of MeritTuple for L1Proc_a 4.6-DEV test (Carmelo)
I made the usual comparison of number of events with different value in MeritTuple wrt P202. The fix introduced in this release improves a lot the TKR part, but not enough to declare victory: there are still many (13427) events with different TKR direction. In the majority of these (96%) CAL cntr and axis is identical to p202.
Here a few summary plots: 
I also tried to count the event that change class: 177 out of 2665218 change classification (out of diagonal in table below):
L1\P202 |
Transient |
Source |
Clean |
UltraClean |
Other |
Total |
|---|---|---|---|---|---|---|
Transient |
77567 |
5 |
2 |
2 |
74 |
77650 |
Source |
3 |
5217 |
0 |
1 |
1 |
5222 |
Clean |
1 |
0 |
1422 |
2 |
0 |
1425 |
UltraClean |
1 |
2 |
3 |
7163 |
0 |
7169 |
Other |
79 |
0 |
0 |
1 |
2273672 |
2573752 |
Total |
77651 |
5224 |
1427 |
7169 |
2573747 |
2665218 |
After discussion at C&A meeting I tried to make a few more high-level tests, comparing events in gamma-ray classes (in either lp60 or P202) and the fraction of events with difference in energy or angle larger than some threshold.
Direction P202-lp60 angle:
Class |
Tot |
log(Angle)>-3.40 (~0.023 deg) |
|---|---|---|
Transient |
91546 |
259 (0.3 %) |
Source |
13826 |
13 (0.1 %) |
Clean |
8600 |
5 (0.1 %) |
UltraClean |
7175 |
4 (0.1 %) |
Plot: |
|
|
Energy P202-lp60 ratio:
Class |
Tot |
|L1.E/E-1|>1.0% |
|L1.E/E-1|>5.0% |
|L1.E/E-1|>10.0% |
|---|---|---|---|---|
Transient |
91546 |
257 (0.3 %) |
124 (0.1 %) |
76 (0.001 %) |
Source |
13826 |
56 (0.4 %) |
28 (0.2 %) |
12 (0.001 %) |
Clean |
8600 |
39 (0.5 %) |
23 (0.3 %) |
10 (0.0012 %) |
UltraClean |
7175 |
38 (0.5 %) |
23 (0.3 %) |
10 (0.0014 %) |
Plot: |
|
|
|
|
Variations seem quite small. Large change in Energy are due change in Energy Method
Another comparison between lp60 and P202
Here is the log10(angular separation) plot for lp60:
As Carmelo just posted, the improvement for the tkr variables is fantastic!
MeritTuple test of run 388740704 (Carmelo)
A second run was processed in L1Proc with GR 17-35-24-lp60 in order to have more statistics and do some high-level study with some bright source. Run 388740704 has been chosen at it contains GRB130427A with several high energy photons.
Unfortunately, event if the calibration flavor settings is correct, we used in L1proc test different calibration constants. You can scan the log file for details, here an example with CAL_Ped:
the same query "SELECT ser_no FROM metadata_v2r1 WHERE ((completion="OK") AND (instrument="LAT") AND (calib_type="CAL_Ped") AND (flavor="p7repro") AND ("2013-04-27 07:31:47">vstart) AND ("2013-04-27 07:31:47"<vend) AND ("PROD"=PROC_LEVEL)) ORDER BY update_time desc" returned file "$(LATCalibRoot)/CAL/p7repro/pedavr_382m_386m.xml" in P202 and file "$(LATCalibRoot)/CAL/p7repro/pedavr_388m_392m.xml" in this test. As a consequence, this test is not a comparison of GR lp60 vs gr17 as in previous run, but a real end-to-end test of how this run would look like if we had to reprocess it now (at least for the CAL calib). In other words, it includes the differences in GR and small differences in CAL calibrations.
Here the tables and plots of the fraction of events with difference in energy or angle larger than some threshold. As expected there are many more events now with some difference, the small update in in calibration seem to be the dominant effect.
Direction P202-lp60 angle:
Class |
Tot |
log(Angle)>-3.40 (~0.023 deg) |
|---|---|---|
Transient |
215400 |
20829 (9.7 %) |
Source |
63401 |
1797 (2.8 %) |
Clean |
49960 |
1351 (2.7 %) |
UltraClean |
41597 |
1110 (2.7 %) |
Plot: |
|
|
Energy P202-lp60 ratio:
Class |
Tot |
|L1.E/E-1|>5.0% |
|---|---|---|
Transient |
215400 |
1069 (0.5 %) |
Source |
63401 |
197 (0.3 %) |
Clean |
49960 |
157 (0.3 %) |
UltraClean |
41597 |
126 (0.3 %) |
Plot: |
|
|
Another check is on the high-energy photons found in GRB130427A analysis. I scanned the same events as I did for the GRB see here for details, and observed essentially no differences for these events:
t.Scan("EvtElapsedTime-388741629:CTBBestEnergy:L1.CTBBestEnergy:abs(L1.CTBBestEnergy/CTBBestEnergy-1):(FT1EventClass&0x1d):FT1EventClass==L1.FT1EventClass): CTBCORE:L1.CTBCORE:CTBBestEnergyProb:L1.CTBBestEnergyProb:FT1Ra:L1.FT1Ra:FT1Dec:L1.FT1Dec",
"(EvtEventId==3390429) || (EvtEventId==2464261) || (EvtEventId==3444739) || (EvtEventId==4705506) || (EvtEventId==11983827) || (EvtEventId==2580344) || (EvtEventId==2731009) || (EvtEventId==2963451) || (EvtEventId==3283498) || (EvtEventId==2713633) || (EvtEventId==3268741) || (EvtEventId==2648342) || (EvtEventId==2482436)")
************************************************************************************************************************************************************************************
* Row * EvtElapse * CTBBestEn * L1.CTBBes * abs(L1.CT * (FT1Event * (FT1Event * CTBCORE * L1.CTBCOR * CTBBestEn * L1.CTBBes * FT1Ra * L1.FT1Ra * FT1Dec * L1.FT1Dec *
************************************************************************************************************************************************************************************
* 472948 * 19.064351 * 71305.953 * 71213.570 * 0.0012955 * 0 * 1 * 0 * 0 * 0.6706129 * 0.6701418 * 171.99769 * 171.99771 * 31.862634 * 31.862632 *
* 476376 * 23.645536 * 10775.267 * 10780.360 * 0.0004726 * 29 * 1 * 0.0445221 * 0.0445221 * 0.8618040 * 0.8618040 * 172.17590 * 172.15327 * 27.288623 * 27.295263 *
* 494903 * 48.007806 * 28708.009 * 28603.675 * 0.0036343 * 5 * 1 * 0.6221749 * 0.5748100 * 0.2407225 * 0.2446980 * 173.28053 * 173.2827 * 27.651189 * 27.650981 *
* 507524 * 64.894754 * 11988.679 * 11930.457 * 0.0048564 * 29 * 1 * 0.8034922 * 0.8034922 * 0.3546319 * 0.3373169 * 173.31433 * 173.31712 * 27.725812 * 27.723766 *
* 519623 * 80.940313 * 13035.263 * 13009.251 * 0.0019955 * 5 * 1 * 0.8435378 * 0.8435378 * 0.2496242 * 0.2447519 * 173.25058 * 173.24089 * 27.677135 * 27.680463 *
* 522826 * 85.159673 * 27310.296 * 27199.644 * 0.0040516 * 5 * 1 * 0.1386326 * 0.0806780 * 0.2626658 * 0.2752659 * 173.39186 * 173.46028 * 27.820808 * 27.834432 *
* 566726 * 141.52865 * 25199.880 * 25200.130 * 9.920e-06 * 5 * 1 * 0.6293173 * 0.5874550 * 0.7047906 * 0.7251923 * 173.22123 * 173.22604 * 27.811012 * 27.812912 *
* 627305 * 214.35738 * 11977.75 * 11974.966 * 0.0002323 * 29 * 1 * 0.8143669 * 0.8454899 * 0.8455054 * 0.8470528 * 172.80119 * 172.80377 * 27.971977 * 27.975347 *
* 630337 * 217.88755 * 15591.344 * 15570.979 * 0.0013061 * 29 * 1 * 0.5907437 * 0.6044979 * 0.6589291 * 0.6541520 * 173.06861 * 173.06947 * 27.659246 * 27.660011 *
* 652206 * 243.54966 * 98431.859 * 98157.437 * 0.0027879 * 29 * 1 * 0.6527069 * 0.6527069 * 0.7233405 * 0.7233405 * 173.09648 * 173.09504 * 27.704078 * 27.704475 *
* 663476 * 256.70178 * 49680.613 * 49719.105 * 0.0007747 * 5 * 1 * 0.7985399 * 0.8340377 * 0.7023981 * 0.6979178 * 173.19381 * 173.19599 * 27.637081 * 27.639444 *
* 937454 * 611.00936 * 43991.320 * 44033.296 * 0.0009542 * 29 * 1 * 0.9545454 * 0.9890909 * 0.7674124 * 0.7106212 * 173.17210 * 173.17063 * 27.676036 * 27.674320 *
* 2552075 * 3410.2581 * 39623.308 * 39602.593 * 0.0005227 * 29 * 1 * 0.8786774 * 0.9299595 * 0.7597252 * 0.7537148 * 173.13639 * 173.13305 * 27.720428 * 27.718521 *
************************************************************************************************************************************************************************************
==> 13 selected entries
Event class table:
L1\P202 |
Transient |
Source |
Clean |
UltraClean |
Other |
Total |
|---|---|---|---|---|---|---|
Transient |
149982 |
137 |
153 |
97 |
990 |
151359 |
Source |
123 |
13162 |
2 |
15 |
14 |
13316 |
Clean |
152 |
4 |
8031 |
243 |
15 |
8445 |
UltraClean |
82 |
14 |
216 |
40892 |
26 |
41230 |
Other |
1027 |
5 |
6 |
12 |
3108814 |
3109864 |
Total |
151366 |
13322 |
8408 |
41259 |
3109859 |
3324214 |
Comparison of AllGamma simulation
AG-GR-v17r35p24lp60-SL6 done in Lyon (JIRA SIMREQ-134 , DataCatalog link ) compared with AG-GR-v17r35p24lp43-allE-2 used for IRF P7Rep V15.
I checked the faction of events that survive the Transient, Source and Clean cut (plots below). The differences are very small (~5% at 100 MeV). I think it is safe to use this GR in L1Proc.
|
|
|---|
Philippe made a direct comparison between data and MC efficiencies (from Transient to Clean), observing same behavior between lp60 and lp43:
