status: Complete
last update: 6 Aug 2012
This page is a record of the configuration for the P130 reprocessing project, recalculating the diffuse response (see here for a description of the change). This task is identical with P120-FT1, v2.1 [please see for details], run in July 2011, except for the following details.
The data sample for P130 reprocessing includes:
First run |
239557414 (MET), 2008-08-04 15:43:34 (UTC) |
beginning of Science |
Last run |
356429251 (MET), 2012-04-18 08:07:29 (UTC) |
|
Total runs |
20,385 |
|
Total input MERIT events |
44,398,334,636 |
all "events" |
Total disk space used |
33.9 TB |
|
Summary from DataCatalog as of 4/17/2012 and updated 8/6/2012 (after new evt selection filter).
Name |
Files |
Events |
Size |
---|---|---|---|
FT1 |
20385 |
184,008,254 |
16.6 GB |
LS1 |
20385 |
1,295,164,924 |
210.5 GB |
ELECTRONFT1 |
20385 |
85,057,599 |
8.0 GB |
EXTENDEDFT1 |
20385 |
11,240,869,507 6,280,448,356 |
984.6 550.3GB |
EXTENDEDLS1 |
20385 |
11,240,869,507 6,280,448,356 |
1.8 1.0 TB |
This task generates all desired FITS data products. An example of the code processing chain appears on a child page.
|
MET(sec) |
MJD(days) |
UNX(sec) |
UTC |
---|---|---|---|---|
1st run |
333886531 |
55774.427442129629 |
1312193731.000 |
2011-08-01 10:15:29 |
last run |
349266293 |
55952.433946759258 |
1327573493.000 |
2012-01-26 10:24:51 |
|
Run |
Task Stream |
---|---|---|
Start |
333886531 |
16460 |
End |
356429251 |
20384 |
Task Location |
/nfs/farm/g/glast/u38/Reprocess-tasks/P130-FT1 |
Task Status |
http://glast-ground.slac.stanford.edu/Pipeline-II/task.jsp?task=74765664 |
Input Data |
MERIT from P120-MERIT (runs 239557414 - 333880535) and Level 1 (runs 333886531 - present) |
spacecraft data |
FT2 from P105 (runs 239557414 - 271844560), then from current Level 1 production |
Input Run List |
ftp://ftp-glast.slac.stanford.edu/glast.u38/Reprocess-tasks/P130-FT1/config/runFile.txt |
Reprocessing Mode |
reFT1 |
meritFilter |
FT1EventClass!=0 for P120-reprocessing, and (FT1EventClass& 0x00003EFF)!=0 for subsequent data |
evtClassDefs |
00-19-01 |
eventClassMap |
EvtClassDefs_P7V6.xml |
ScienceTools |
09-26-02 |
Code Variants |
redhat5-i686-32bit-gcc41 (Optimized) |
Diffuse Model |
based on contents of /afs/slac.stanford.edu/g/glast/ground/GLAST_EXT/diffuseModels/v2r0 ) |
Diffuse Response |
'source' using P7SOURCE_V6 IRF |
IRFs |
P7V6, contained within ScienceTools release |
Output Data Products |
Processing chain for FITS data products
Data Product |
selection |
makeFT1 |
gtdiffrsp |
gtmktime |
gtltcube |
---|---|---|---|---|---|
FT1 (for FSSC) |
'source' and above |
||||
LS1 (for FSSC) |
'transient' and above |
||||
FT1EXTENDED |
FT1EventClass!=0 |
||||
LS1EXTENDED |
FT1EventClass!=0 |
||||
ELECTRONFT1 |
CTBParticleType==1 |
Note that diffuse response is calculated for 'source' and 'clean' event classes only.
Note on 'Code Variant': The SLAC batch farm contains a mixture of architectures , both hardware (Intel/AMD 64-bit) and software (RHEL5-64, gcc v4.1, etc.).
With a sample of the first 8128 runs, the CPU-intensive part of the task (mergeClumps) is consuming ~42 CPU-min/job (differences in machine class are small). TrickleStream configured to allow 1500 simultaneous mergeClumps jobs, and up to 150 job starts every 200 second cycle.