P116 Reprocessing
status: In Progress Complete
last update: 11 17 November 2010
This page is a record of the configuration for the P116 reprocessing project, rebuilding FT1 files (and friends) for all of science data with new, extended event classifications.
...
| Code Block |
|---|
Data/Flight/Reprocess/P116:FT1 r0239557414 Data/Flight/Reprocess/P116:LS1 r0239557414 |
Data Sample
The currently defined data sample for P116 reprocessing includes the following runs.
Block | First run | First run | Last run | Last run | #Runs | #Merit Evts | DataCatalog Source | Note | |||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
1 | 239557414 | 2008-08-04 15:43:34 | 242047683 | 2008-09-02 11:28:02 | 431 | 909,050,672 | /Data/Test/Flight/Repro/ReproTest7 | reprocessed for alignment fix | |||||
1 | 242053458 | 2008-09-02 13:04:17 | 307579060 | 2010-09-30 22:37:38 | 11413 | 24,906,606,023 | /Data/Flight/Level1/LPA/ | Standard Level 1 output (ignore StdIntent) | |||||
1 Subtotal |
|
|
|
| 11844 | 25,815,656,695 |
|
| |||||
2 | 307585048 | 2010-10-01 00:17:26 | 311108362 | 2010-11-10 18:59:20 | 617 | 1,368,042,493 | /Data/Flight/Level1/LPA/ | Standard Level 1 output | |||||
3 |
|
|
|
|
| 311112219 | 2010-11-10 20:03:37 | 311669624 | 2010-11-17 06:53:42 | 98 | 215,382,591 | /Data/Flight/Level1/LPA/ | Standard Level 1 output |
Grand Total |
|
|
|
| 239557414 | 2008-08-04 15:43:33 | 311669624 | 2010-11-17 06:53:42 | 12559 12461 | 27,183399,699081,188779 |
|
|
Final number of selected photon events (in the FT1 and LS1 files) = 435,801,176
These It is expected that a backfill operation will happen to mesh with Level 1 once it begins to produce these new event classifications. Ultimately, these data are destined for the FSSC and were made public as of 17 November 2010
These data (along with subsequent Level 1 data) are in the SLAC Astroserver as P6_public_v2.
Bookkeeping
- (This page): Define ingredients of reprocessing (processing code/configuration changes)
- Processing History database: http://glast-ground.slac.stanford.edu/HistoryProcessing/HProcessingRuns.jsp?processingname=P116
- List of all reprocessings
- List of all data runs reprocessed
- Pointers to all input data files (-> dataCatalog)
- Pointers to associated task processes (-> Pipeline II status)
- Data Catalog database: http://glast-ground.slac.stanford.edu/DataCatalog/folder.jsp
- Lists of and pointers to all output data files
- Meta data associated with each output data product
P116-FT1
| Anchor | ||||
|---|---|---|---|---|
|
(This task is roughly equivalent to the P105-FT1 task but with updates and modernization.)
Status chronology
- 11/17/2010 - reprocess final backfill block3 (98 runs)
- 11/12/2010 - discover that gtdiffrsp params were incorrect (evtclass parm ignored, so all events got all diff rsp calc). Next (last) backfill block will be corrected.
- 11/11/2010 - gear up for reprocessing data block2 (617 runs)
- 10/18/2010 - 146 mergeClumps jobs ran out of CPU time. Changed to xxl batch queue and rolled back.
- 10/14/2010 - Validation complete, begin production for data block1
- 10/1/2010 - Begin task construction
...
Task Location | /nfs/farm/g/glast/u38/Reprocess-tasks/P116-FT1 |
Task Status | http://glast-ground.slac.stanford.edu/Pipeline-II/index.jsp |
Input Data Selection | MERIT |
Input Run List | ftphttp://ftpglast-glastground.slac.stanford.edu/glast.Decorator/Decorate/u38/Reprocess-tasks/P116-FT1/config/runFile.txt |
Reprocessing Mode | reFT1 |
evtClassDefs | 00-18-00 |
photon cut | pass6_FSW_cuts |
eventClassifier | dataclean_classifier.py |
eventClassMap | n/a |
s/c data | FT2 from P105 (runs 239557414 - 271844560), then from current Level 1 production |
ScienceTools | 09-18-05 (SCons build) |
Allowed Code Variants | redhat4-i686-32bit-gcc34, redhat4-x86_64-64bit-gcc34, |
Diffuse Model | /afs/slac.stanford.edu/g/glast/ground/releases/analysisFiles/diffuse/v2/source_model_v02.xml ) |
Diffuse Response IRFs | P6_V3_DIFFUSE, P6_V3_DATACLEAN |
Output Data Products |
...