...
P120-FT1
| Anchor | ||||
|---|---|---|---|---|
|
This task will be run twice: Pass 1 will perform event classification for source and transient events and allow analysis to produce diffuse class IRFs; Pass 2 will be identical to Pass 1 but will include diffuse classification. The latest word from C&A is that diffuse response will only be calculated for 'source' class events.
Status chronology
- 8/31/2010 - Pass 1 of this task is complete (through 31 July 2010)
- 8/30/2010 - Problem with makeFT1 stressing /u38. Jim makes update to fitsGenApps => ST 09-18-03, put into production at stream 1400.
- 8/29/2010 - Begin Pass 1 of task...
Configuration
Task Location | /nfs/farm/g/glast/u38/Reprocess-tasks/P120-FT1 |
Task Status | http://glast-ground.slac.stanford.edu/Pipeline-II/index.jsp |
Input Data Selection | MERIT (from P120-MERIT) |
Input Run List | ftp://ftp-glast.slac.stanford.edu/glast.u38/Reprocess-tasks/P120-FT1/config/runFile.txt |
Reprocessing Mode | reFT1 |
evtClassDefs | 00-17-00 |
meritFilter | FT1EventClass!=0 |
eventClassifier | Pass7_Classifier.py (OBSOLETE) |
eventClassMap | EvtClassDefs_P7V3.xml (in evtUtils) |
s/c data | FT2 from P105 (runs 239557414 - 271844560), then from current Level 1 production |
ScienceTools | 09-18-01 through stream 1399, then 09-18-03 (SCons build) |
Code Variants | redhat4-i686-32bit-gcc34 and redhat5-i686-32bit-gcc41 (Optimized) |
Diffuse Model | /afs/slac.stanford.edu/g/glast/ground/releases/analysisFiles/diffuse/v2/source_model_v02.xml ) |
Diffuse Response IRFs | P7trans_v3mc, P7source_v3mc (TEMPORARY) |
IRFs | implemented as 'custom irf', files in /afs/slac.stanford.edu/g/glast/ground/PipelineConfig/IRFS/Pass7.3 |
Output Data Products |
...
Data Product | makeFT1 | gtdiffrsp | gtmktime | gtltcube |
|---|---|---|---|---|
FT1 | true | true for | true | false(NEEDS TO CHANGE) |
LS1 | true | false | true | false |
ELECTRONFT1 | true | false | true | false |
Note that diffuse response is calculated only for 'source' class events (and not transient); diffuse class events are not yet classified.
Note on 'Code Variant': The SLAC batch farm contains a mixture of architectures , both hardware (Intel/AMD 64-bit) and software (RedHat Enterprise Linux 4, and 5, gcc 3.4, 4.1, etc.).
...
- 8/31/2010 - With P120-MERIT files nicely distributed across xroot servers, there were no xroot limitations to the processing. After the update to makeFT1, there was no longer an issue with overloading /u38 ($PWD). The next bottleneck was the pipeline processing itself. This task consists of three batch jobs and four scriptlets; it was observed that the pipeline allowed hundreds of jobs to dwell in the READY state for extended periods of time, thus making it impossible to keep LSF saturated. Nevertheless, the maximum number of simultaneous jobs approached 2000. The task essentially completed in 6 8 hours, although some lingerers kept 'running' for another nine hours (mostly in SSUSP). A profile of job processing rate appears in this plot:
...