status: In Progress
last update: 17 Oct 05 Nov 2013
This page is a record of the configuration and execution of the P202 reprocessing project, full reprocessing from DIGIs using Pass7 analysis code. This project involves reprocessing with Pass7 classification trees and up-to-date alignment/calibration data. This task will read DIGI files and emit RECON, MERIT, GCR and CAL ROOT files, and the standard array of FITS files. It will be a CPU-intensive and storage-intensive enterprise requiring months of elapsed time and of order 0.7 Pbyte of storage. At the time of this task beginning, there will be about 20,000 science runs in Fermi (3.5 years accumulation).
...
- 9/17/2013
- 13:33 Begin block 11 backfill, 539 runs in the range 398087830 through 401106634 (2013-09-17 11:48:16). Once complete, 28412 runs will have been reprocessed.
- 9/19/2013
- 08:30 block 11 complete. The bulk of runs completed within 14 hours, but stragglers, failed/terminated jobs, due to transient problems, bad batch machines, etc., delayed completion until this morning. DataCatalog reports a total of 61,827,628,706 events in {CAL,MERIT,RECON,GCR} and 8,991,163,530 events in FILTEREDMERIT.
- xroot scratch space cleaned up.
- 10/4/2013
- Prepare for block 12 backfill. Warren suggests ending with run 402560477 (2013-10-04 06:21:14 UTC). First run of block 12 is 401112810. There are 263 runs in this backfill block which will bring the reprocessed run total to 28675.
- 10/15/2013
Prepare for block 13 backfill. Warren suggests ending with run 403510814, 170 new runs.
Run
Run start time (UTC)
Task Stream
Block 13 start
402566464
2013-10-04 08:01:01
28675
Block 13 end
403510814
2013-10-15 06:20:11
28844
- 11/05/2013
- Today P7REP goes public.
- Prepare for block 14 backfill – the ultimate backfill !for P7REP. Last run = 405329691 (2013-11-05 07:34:48)
313 new runs to reprocess...
Run
Run start time (UTC)
Task Stream
Block 14 start
403516762
2013-10-15 07:59:19
28845
Block 14 end
405329691
2013-11-05 07:34:48
2915829157
Configuration
Task Location | /nfs/farm/g/glast/u38/Reprocess-tasks/P202-ROOT |
Task Status | |
GlastRelease | 17-35-24-gr17 and 17-35-24-rp04 (SCons RHEL4-32 build) |
Run Selection | based on a modified "standard" selection, see https://confluence.slac.stanford.edu/display/SCIGRPS/Official+LAT+Datasets |
s/c data | "standard" Public Release 2 https://confluence.slac.stanford.edu/display/SCIGRPS/Official+LAT+Datasets |
Input Run List | ftp://ftp-glast.slac.stanford.edu/glast.u38/Reprocess-tasks/P202-ROOT/config/runList.txt |
photonFilter | CTBParticleType==1 && ((FT1EventClass & 0x00003EFF)!=0) |
electronFilter | CTBParticleType==1 |
Code Variants | redhat4-i686-32bit-gcc34 (Optimized) |
jobOpts | ftp://ftp-glast.slac.stanford.edu/glast.u38/Reprocess-tasks/P202-ROOT/config/doRecon.txt |
Output Data Products |
...