...
status: Running
last update: 15 17 March 2011
This page is a record of the configuration for the P120 reprocessing project, event reclassification using Pass 7.3/7.4. This project involves reprocessing with Pass7 classification trees and (ultimately) new IRFs. The name "P120" derives from the word "processing" and the initial file version to be used for the output data products, e.g., r0123456789_v120_merit.root.
...
P120-MERIT
| Anchor | ||||
|---|---|---|---|---|
|
Status chronology
- 3/17/2011 - Last block complete (one instance of skimmer failure in mergeClumps)
- 3/15/2011 - Expand P120 to present (last run 321756673, 2011-03-14 00:51:11 UTC). Three runs continue to be dummied out. Once those runs have proper RECON files, their streams can be rolled back.
run
task stream
# subStreams
306353950
11624
10
316611240
13431
8
320850543
14171
7
- 2/4/2011 - Expand P120 to present (last run 318211122, 2011-01-31 23:58:40 UTC). This include three runs for which there are no Recon files which have, for the moment, been supplied with 'dummy' entries in the runFile.:
306353950
'bad chunk' (known previously) - awaiting GR update
313483912
missing 700s, being worked on
316611240
'bad chunk' - awaiting GR update
- 1/12/2011 - Expand P120 coverage to include Crab ToO
- Crab ToO 9/23/2010 15:50:50 to 9/27/2010 19:49:38, corresponding to MET r0306949696-r0307308940
- reconfig through end of Sep 2010: 10916 -> 11841 runs, increase of 925 runs
- No recon file for run 306353950 (being worked on, for the moment, place dummy file in runFile.txt)
- First run in new block completed, awaiting checkered flag to continue...
- 8/29/2010 - Discovered three merge steps that silently failed (xroot file access). TASK complete.
- 8/28/2010 - processing formally complete (10916 runs), but some discrepancy in # of events
- 8/26/2010 - serious xroot problems. See initial distribution of files across xroot servers. From this report (courtesy Wilko) it is easy to see where problems are likely to arise - when the number of servers involved is small, e.g. two or three.
- 8/19/2010 - production continues at a crawl due to xroot server difficulties
- 8/16/2010 - resume full production, but at a slow trickle (max 350 simultaneous processClump jobs)
- 8/8/2010 - block 2 reprocessing complete. Many xroot server problems. (5 days to process 2084 runs)
- 8/3/2010 - begin block 2 reprocessing (through 255132033 MET), bringing the total runs reprocessed to 2721, about 5-1/2 months of data.
- 7/28/2010 - block 1 re-reprocessing complete
- 7/27/2010 - New GlastRelease (v17r35p10) containing new evtUtils, "to make the FT1EventClass bits compatible with the ScienceTools". Cleanup, including removing all files created last week during the first attempt.
- 7/21/2010 - block 1 reprocessing complete
- 7/20/2010 - agree upon 'pilot block' of runs (239557417 - 243220241), 637 runs. Begin...
- 7/19/2010 - submit first test run. success. await feedback
...
This task will be run twice: Pass 1 will perform event classification for source and transient events and allow analysis to produce diffuse class IRFs; Pass 2 will be identical to Pass 1 but will include diffuse classification. The latest word from C&A is that diffuse response will only be calculated for 'source' class events.
Status chronology
- 3/17/2011 - Catch up with P120-MERIT (last run 321756673, 2011-03-14 00:51:11 UTC)
- 3/15/2011 - Due to missing run, rollback runs 11624-11841. Bookkeeping is now correct.
- 2/4/2011 - Catch up with merit production (through 30 Sep 2010), but with one missing run/stream
- 1/28/2011 - Pass 7.4 reincarnation of this task complete through 31 Jul 2010
- 1/24/2011 - Entire task, xroot files, dataCat entries deleted. Prepare to reprocess as Pass 7.4
- 8/31/2010 - Pass 1 of this task is complete (through 31 July 2010)
- 8/30/2010 - Problem with makeFT1 stressing /u38 (very large temporary file needed when using xml representation of event classes was being written to $PWD). Jim makes update to fitsGenApps => ST 09-18-03, put into production at stream 1400.
- 8/29/2010 - Begin Pass 1 of task...
...