status: Running
last update: 4 5 April 2012
This page is a record of the configuration and execution of the P202 reprocessing project, full reprocessing from DIGIs using Pass7 analysis code. This project involves reprocessing with Pass7 classification trees and new IRFs. This task will read DIGI files and emit RECON, MERIT, GCR and CAL ROOT files, and the standard array of FITS files. It will be a CPU-intensive and storage-intensive enterprise requiring months of elapsed time and of order 1 Pbyte of storage. At the time of this task beginning, there will be about 20,000 science runs in Fermi (3.5 years accumulation).
...
S/W component | bug fix | status |
|---|---|---|
New ROOT version | 5-min 'transaction timeout' triggered by xroot data server reboot | done 4/3/2012 |
New GlastRelease | 1) include new ROOT version (above); 2) exit with non-zero RC on ROOT write error | in progressdone 4/5/2012, GR 17-35-24-rp04 |
New GPL_TOOLS | check size/checksum of file written to xroot with known size/checksum | pending |
Tuned xroot on new Dell servers | silent file truncation when volume fills up JIRA | done 4/4/2012 (100 MB min space limit -> 100 GB; file system space check cadence changed from 10 min to 2 min) |
New xroot client tools | complain when xroot data server fails on write | done 4/3/2012, v3.1.1 |
New TSkim | 1) new ROOT version (above); 2) complain on ROOT write errors | pendingdone 4/5/2012, v08-02-01 |
New xroot redirector | required step toward enabling HPSS staging | done 4/3/2012, v3.1.1 |
Note also that the FILTEREDMERIT files contain 42 more events than the EXTENDEDFT1 files; they should be identical.
- 4/5/2012 - resume task. New goal is entire science dataset.
Configuration
Task Location | /nfs/farm/g/glast/u38/Reprocess-tasks/P202-ROOT |
Task Status | http://glast-ground.slac.stanford.edu/Pipeline-II/index.jsp |
GlastRelease | 17-35-24-gr17 (SCons RHEL4-32 build) |
Run Selection | based on a modified "standard" selection, see https://confluence.slac.stanford.edu/display/SCIGRPS/Official+LAT+Datasets |
s/c data | "standard" Public Release 2 https://confluence.slac.stanford.edu/display/SCIGRPS/Official+LAT+Datasets |
Input Run List | ftp://ftp-glast.slac.stanford.edu/glast.u38/Reprocess-tasks/P202-ROOT/config/runList.txt |
photonFilter | CTBParticleType==1 && ((FT1EventClass & 0x00003EFF)!=0) |
electronFilter | CTBParticleType==1 |
Code Variants | redhat4-i686-32bit-gcc34 (Optimized) |
jobOpts | ftp://ftp-glast.slac.stanford.edu/glast.u38/Reprocess-tasks/P202-ROOT/config/doRecon.txt |
Output Data Products |
...
<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="33112af031416df3-7679676b-4ea54a4e-916b9340-2fb8c425b725e17107825c94"><ac:plain-text-body><![CDATA[ | Data Product | destination | data content [1] | event selection [1] | makeFT1 | gtselect | gtdiffrsp | gtmktime | ]]></ac:plain-text-body></ac:structured-macro> |
|---|---|---|---|---|---|---|---|---|---|
EXTENDEDFT1 | SLAC | FT1variables | ((FT1EventClass & 0x00003EFF)!=0) |
|
|
|
| ||
FT1 | FSSC+SLAC | FT1variables | 'source' and above |
| | (inherited) |
| ||
EXTENDEDLS1 | SLAC | LS1variables | ((FT1EventClass & 0x00003EFF)!=0) |
|
|
|
| ||
LS1 | FSSC+SLAC | LS1variables | 'transient' and above |
|
| (inherited) |
| ||
ELECTRONFT1 | SLAC | FT1variables | CTBParticleType==1 |
|
|
|
|
...