P130 Reprocessing
status: In Development
last update: 25 January 2012
This page is a record of the configuration for the P130 reprocessing project, recalculating the diffuse response (see here for a description of the change). This task is identical with P120-FT1 [please see for details], run in July 2011, except for the following details.
- New version of ScienceTools (09-26-92) which includes new integration scheme for gtdiffrsp
- New run list (updated for data processed since the end of P120)
- Name of Pipeline task (P120->P130) as well as in file naming
=============== WHAT FOLLOWS NEEDS TO BE UPDATED FOR P130 ====================
Data Sample
The currently defined data sample for P120 reprocessing includes:
First run |
239557414 (MET), 2008-08-04 15:43:34 (UTC) |
beginning of Science |
Last run |
333880535 (MET), 2011-08-01 08:35:33 (UTC) |
Official Pass7 release |
Total runs |
16,459 |
|
Total MERIT events |
35,921,666,747 |
all "events" |
Total FILTEREDMERIT/EXTENDEDFT1/LS1 events |
5,035,929,409 |
all photon event classes |
Total ELECTRONMERIT/ELECTRONFT1 events |
68,055,849 |
|
Total LS1 (FSSC selection) events |
1,025,359,231 |
event classes (bits) 0,2,3,4 (transient, source, clean, ultraclean) |
Total FT1 (FSSC selection) events |
142,042,060 |
event classes (bits) 2,3,4 (source, clean, ultraclean) |
Total disk space used |
33.9 TB |
|
Summary from DataCatalog as of 8/2/2011.
Name |
Files |
Events |
Size |
---|---|---|---|
ELECTRONFT1 |
16459 |
68,055,849 |
6.4 GB |
ELECTRONMERIT |
16459 |
68,055,849 |
147.4 GB |
EXTENDEDFT1 |
16459 |
5,035,929,409 |
441.3 GB |
EXTENDEDLS1 |
16459 |
5,035,929,409 |
816.5 GB |
FILTEREDMERIT |
16459 |
5,035,929,409 |
4.0 TB |
FT1 |
16459 |
142,042,060 |
12.9 GB |
LS1 |
16459 |
1,025,359,231 |
166.6 GB |
MERIT |
16459 |
35,921,666,747 |
28.3 TB |
=============== THE FOLLOWING HAS BEEN UPDATED FOR P130 ====================
Bookkeeping
- (This page): Define ingredients of reprocessing (processing code/configuration changes)
- Processing History database: http://glast-ground.slac.stanford.edu/HistoryProcessing/HProcessingRuns.jsp?processingname=P130
- List of all reprocessings
- List of all data runs reprocessed
- Pointers to all input data files (-> dataCatalog)
- Pointers to associated task processes (-> Pipeline II status)
- Data Catalog database: http://glast-ground.slac.stanford.edu/DataCatalog/folder.jsp
- Lists of and pointers to all output data files
- Meta data associated with each output data product
P130-FT1
This task generates all desired FITS data products. An example of the code processing chain appears on a child page.
Status chronology
- 1/26/2012 - Initial task configured, 1000 runs reprocessed. On hold for validation.
Configuration (version 2)
Task Location |
/nfs/farm/g/glast/u38/Reprocess-tasks/P130-FT1 |
Task Status |
http://glast-ground.slac.stanford.edu/Pipeline-II/ |
Input Data |
MERIT (from P120-MERIT and Level 1) |
spacecraft data |
FT2 from P105 (runs 239557414 - 271844560), then from current Level 1 production |
Input Run List |
ftp://ftp-glast.slac.stanford.edu/glast.u38/Reprocess-tasks/P130-FT1/config/runFile.txt |
Reprocessing Mode |
reFT1 |
meritFilter |
FT1EventClass!=0 |
evtClassDefs |
00-19-01 |
eventClassMap |
EvtClassDefs_P7V6.xml |
ScienceTools |
09-26-02 |
Code Variants |
redhat5-i686-32bit-gcc41 (Optimized) |
Diffuse Model |
based on contents of /afs/slac.stanford.edu/g/glast/ground/GLAST_EXT/diffuseModels/v2r0 ) |
Diffuse Response |
'source' using P7SOURCE_V6 IRF |
IRFs |
P6V7, contained within ScienceTools release |
Output Data Products |
Processing chain for FITS data products
Data Product |
selection |
makeFT1 |
gtdiffrsp |
gtmktime |
gtltcube |
---|---|---|---|---|---|
FT1 |
'source' and above |
true |
true |
true |
false |
LS1 |
'transient' and above |
true |
true |
true |
false |
FT1EXTENDED |
FT1EventClass!=0 |
true |
true |
true |
false |
LS1EXTENDED |
FT1EventClass!=0 |
true |
true |
true |
false |
ELECTRONFT1 |
CTBParticleType==1 |
true |
false |
true |
false |
Note that diffuse response is calculated for 'source' and 'clean' event classes only.
Note on 'Code Variant': The SLAC batch farm contains a mixture of architectures , both hardware (Intel/AMD 64-bit) and software (RHEL5-64, gcc v4.1, etc.).