status: In Progress Complete
last update: 17 Oct 07 Nov 2013
This page is a record of the configuration and execution of the P202 reprocessing project, full reprocessing from DIGIs using Pass7 analysis code. This project involves reprocessing with Pass7 classification trees and up-to-date alignment/calibration data. This task will read DIGI files and emit RECON, MERIT, GCR and CAL ROOT files, and the standard array of FITS files. It will be a CPU-intensive and storage-intensive enterprise requiring months of elapsed time and of order 0.7 Pbyte of storage. At the time of this task beginning, there will be about 20,000 science runs in Fermi (3.5 years accumulation).
...
- 9/17/2013
- 13:33 Begin block 11 backfill, 539 runs in the range 398087830 through 401106634 (2013-09-17 11:48:16). Once complete, 28412 runs will have been reprocessed.
- 9/19/2013
- 08:30 block 11 complete. The bulk of runs completed within 14 hours, but stragglers, failed/terminated jobs, due to transient problems, bad batch machines, etc., delayed completion until this morning. DataCatalog reports a total of 61,827,628,706 events in {CAL,MERIT,RECON,GCR} and 8,991,163,530 events in FILTEREDMERIT.
- xroot scratch space cleaned up.
- 10/4/2013
- Prepare for block 12 backfill. Warren suggests ending with run 402560477 (2013-10-04 06:21:14 UTC). First run of block 12 is 401112810. There are 263 runs in this backfill block which will bring the reprocessed run total to 28675.
- 10/15/2013
Prepare for block 13 backfill. Warren suggests ending with run 403510814, 170 new runs.
Run
Run start time (UTC)
Task Stream
Block 13 start
402566464
2013-10-04 08:01:01
28675
Block 13 end
403510814
2013-10-15 06:20:11
28844
- 11/05/2013
- Today P7REP goes public.
- Prepare for block 14 backfill – the ultimate backfill !for P7REP. Last run = 405329691 (2013-11-05 07:34:48)
313 new runs to reprocess...
Run
Run start time (UTC)
Task Stream
Block 14 start
403516762
2013-10-15 07:59:19
28845
Block 14 end
405329691
2013-11-05 07:34:48
29158
Configuration
29157
- 8/19/2014
- User reported problem with an event inside a ~195s interval with LIVETIME=0 in run 395891323 (2013-07-19 01:48:40 UTC). Problem traced to a bad FT2SECONDS file. Reprocessing took place 14 Aug 2013 using v002 FT2 file. Then on 19 Aug 2013, the FT2 file was updated to v003. M.E. has rebuilt a proper FT2 file in the Reprocess/P202 area, v203 and will roll back entire stream 27487 to rebuild ROOT files. To get this rollback to run, the following steps were necessary:
config.py - change ft2Selection from 'P105' to 'P202' (commonTools/repTools.py:findFT2() already knows about P202, so no changes needed there.)
commonTools/setupGR.sh - change svsopts from 'redhat5' back to 'redhat4' (TEMPORARILY) for processClump step
commonTools/setupSkimmer.py - a series of reversions:
- revert to skimmer 08-02-01
- revert to rootVersion = 'v5.26.00a-gl6'
- revert glastBuild from redhat5-i686-32bit-gcc41 to
redhat4-i686-32bit-gcc34 to get libraries for mergeClumps step.
- revert to old ROOTSYS definition that includes compiler version as last path item
- The mods to setupGR.sh and setupSkimmer.py have been preserved as OLD-setupGR.sh and OLD-setupSkimmer.py in the commonTools directory
- User reported problem with an event inside a ~195s interval with LIVETIME=0 in run 395891323 (2013-07-19 01:48:40 UTC). Problem traced to a bad FT2SECONDS file. Reprocessing took place 14 Aug 2013 using v002 FT2 file. Then on 19 Aug 2013, the FT2 file was updated to v003. M.E. has rebuilt a proper FT2 file in the Reprocess/P202 area, v203 and will roll back entire stream 27487 to rebuild ROOT files. To get this rollback to run, the following steps were necessary:
Configuration
Task Location | /nfs/farm/g/glast/u38/Reprocess-tasks/P202-ROOT |
Task Status | |
GlastRelease | 17-35-24-gr17 and 17-35-24-rp04 (SCons RHEL4-32 build) |
Run Selection | based on a modified "standard" selection, see https://confluence.slac.stanford.edu/display/SCIGRPS/Official+LAT+Datasets |
Task Location | /nfs/farm/g/glast/u38/Reprocess-tasks/P202-ROOT |
Task Status | |
GlastRelease | 17-35-24-gr17 and 17-35-24-rp04 (SCons RHEL4-32 build) |
Run Selection | based on a modified "standard" selection, see https://confluence.slac.stanford.edu/display/SCIGRPS/Official+LAT+Datasets |
s/c data | "standard" Public Release 2 https://confluence.slac.stanford.edu/display/SCIGRPS/Official+LAT+Datasets |
Input Run List | ftp://ftp-glast.slac.stanford.edu/glast.u38/Reprocess-tasks/P202-ROOT/config/runList.txt |
photonFilter | CTBParticleType==1 && ((FT1EventClass & 0x00003EFF)!=0) |
electronFilter | CTBParticleType==1 |
Code Variants | redhat4-i686-32bit-gcc34 (Optimized) |
jobOpts | ftp://ftp-glast.slac.stanford.edu/glast.u38/Reprocess-tasks/P202-ROOT/config/doRecon.txt |
Output Data Products |
...
- 3/2/2012 - Define block 1 as the 776 runs in P202-ROOT block 1. Configure trickleStream and begin (14:08)
- 3/31/2012 - Define block 2 as 5600 runs. Reconfig trickleStream and begin (18:05)
- 4/01/2012 - Block 2 complete (most of the 4824 jobs completed in about six hours w/1000 job limit).
- 5/31/2012
- Discover stream 5599 (run 271999199) requires rollback - new MERIT file version (v203->v205).
- Regenerate runlist with 20,229 input MERIT files.
- Minor config change: twoClumpMin=False (formerly True, but caused unnecessary extra processClump dummy jobs)
- 6/5/2012 - Final cleanup
Five runs are responsible for discrepant event tallies:
Run
Stream
Reason
Action
239557414
0
bad TCut
rollback ROOT + FITS
241599746
352
bad processing order
rollback FITS
245403855
1019
bad start time
change start time in runList and rollback
332661583
16244
silent root errors
rollback ROOT
339081502
17416
silent root errors
rollback ROOT
Code changes:
directory
script
modification
commonTools/00-01-00
repTools.py
added new getKey() function to extract #events from FITS event files
P202-FITS/config
config.py
added os.environ['HEADASNOQUERY']='true' to enable 'ftlist' to run in batch
P202-FITS/config
makeFITS.py
add diagnostic print of #evts in FITS files for each processing sub-step
After these actions, the dataCatalog tallies are now consistent:
Name
Type
Files
Events
Size
Created (UTC)
Links
Group
20229
0
8.5 GB
02-Mar-2012 00:06:07
Group
20229
90,904,582
205.7 GB
25-Jan-2012 00:53:32
Group
20229
189,323,074
17.8 GB
02-Mar-2012 00:06:06
Group
20229
1,325,204,821
215.3 GB
02-Mar-2012 00:06:08
Group
20229
6,291,424,926
574.7 GB
02-Mar-2012 00:06:09
Group
20229
6,291,424,926
1,020.1 GB
02-Mar-2012 00:06:09
Group
20229
6,291,424,926
5.3 TB
25-Jan-2012 00:53:29
Group
20229
44,125,679,961
35.4 TB
25-Jan-2012 00:53:30
Group
20229
44,125,679,961
590.0 TB
25-Jan-2012 00:53:33
Group
20229
44,125,679,961
942.7 GB
25-Jan-2012 00:53:31
Group
20229
44,125,679,961
128.7 TB
25-Jan-2012 00:53:31
Note that the number of events in ELECTRONFT1 files are not currently tallied by the dataCatalog.
- 8/24/2012 - Configure and run backfill through 31 July 2012
10/7/2012 - Rollback the following seven streams to fix corrupt FITS files. The makeFT1 app received a large number of ROOT error, but terminated normally. The resulting FT1/LS1 files had multiple symptoms, including one discovered by the FSSC during transfer to them: one EVENT_ID was repeated ~1000 times. Scan of log files uncovered an additional five mergeClumps with these errors.
Code Block %INFO: 20120331:20:34:32 - makeFITS(run)/line-127 - Running: makeFT1 ---------------- start commentary ---------------- About to run [time makeFT1 rootFile=/scratch/glastmp/P202-FITS/3345/r0259101994_v202_merit.root fitsFile=/scratch/glastmp/P202-FITS/3345/gll_xp_p202_r0259101994_v202.fit TCuts=/afs/slac.stanford.edu/g/glast/ground/releases//volume04/evtClassDefs/00-19-04/data/pass7.6_Extended_cuts_L1 dict_file=/afs/slac.stanford.edu/g/glast/ground/releases//volume04/evtClassDefs/00-19-04/data/FT1variables tstart=259101996.933000 tstop=259106272.085000 file_version=202 tempRootFile=/scratch/glastmp/P202-FITS/3345/dummy.root xml_classifier=/afs/slac.stanford.edu/g/glast/ground/releases//volume04/evtClassDefs/00-19-04/xml/EvtClassDefs_P7V6.xml evtclsmap=FT1EventClass chatter=4 debug=yes] at Sat Mar 31 20:34:32 2012 ---------------- start log ---------------- This is makeFT1 version ScienceTools-09-27-01 applying TCut: ((FT1EventClass & 0x00003EFF)!=0) && (EvtElapsedTime >= 259101996) && (EvtElapsedTime <= 259106273)) Warning in <TClass::TClass>: no dictionary for class FileHeader is available Warning in <TClass::TClass>: no dictionary for class FileHeaderRootObj<int> is available Warning in <TClass::TClass>: no dictionary for class RootObj<int> is available R__unzip: error in header Error in <TBasket::ReadBasketBuffers>: fNbytes = 15550, fKeylen = 86, fObjlen = 31912, noutot = 0, nout=0, nin=11010118, nbuf=7864444 Error in <TBranch::GetBasket>: File: /scratch/glastmp/P202-FITS/3345/r0259101994_v202_merit.root at byte:459100837, branch:Vtx2LongDoca, entry:510338, badread=0, nerrors=1, basketnumber=1 [...]
Stream
Run
3345
259101994
<- found by FSSC
4122
263571912
4707
266893978
13927
319436826
16181
332306548
<- found by FSSC
17430
339161346
17479
339408141
- 10/15/2012 - Reconfigure for block 4 backfill and begin trickleStream.
- Discover run 22240 makeFT1 contains error messages as the mentioned last week (above). Rolling back the entire chain of processing, starting with P202-ROOT seemed to do the trick.
- 10/17/2012 - block4 backfill complete.
12/18/2012 - Prepare for block 5. Before:
Name
Type
Files
Events
Size
EXTENDEDFT1
Group
23141
7,252,996,601
662.6 GB
EXTENDEDLS1
Group
23141
7,252,996,601
1.1 TB
FT1
Group
23141
229,126,932
21.5 GB
LS1
Group
23141
1,549,133,207
251.7 GB
12/19/2012 - block5 backfill complete. Final DataCatalog numbers:
Name
Type
Files
Events
Size
EXTENDEDFT1
Group
24142
7,578,468,156
692.3 GB
EXTENDEDLS1
Group
24142
7,578,468,156
1.2 TB
FT1
Group
24142
240,949,332
22.6 GB
LS1
Group
24142
1,621,773,636
263.5 GB
- 3/6/2013 - Many changes!
- New task created (version 0.9 -> 1.0)
- Update to run native on RHEL5-64 and RHEL6-64
- New Interstellar Emission Model, v2r0 -> v3r0, provided by Luigi Tibaldo.
- Update ScienceTools from 09-27-01 to 09-31-01
- Update IRFS from P7*V6 to P7REP_V10 ( = CLEAN or SOURCE)
- This means that rather than rolling-back, the task starts from scratch.
- New evtClassDiffs version: 00-19-05, which changes P7V6 -> P7REP, the new designation.
Reference list of all existing FITS files (with old diffuse model) generated and stored here: /nfs/farm/g/glast/u38/Reprocess-tasks/P202-FITS/config/task-v0.9
Code Block diffmodel_p7rep_clean_v10_reduced.xml diffmodel_p7rep_clean_v10.xml diffmodel_p7rep_source_v10_reduced.xml diffmodel_p7rep_source_v10.xml gal_p7rep_v10_v1.fits gal_p7rep_v10_v1_reduced.fits iso_p7rep_clean_v10_back_v1.txt iso_p7rep_clean_v10_front_v1.txt iso_p7rep_clean_v10_v1.txt iso_p7rep_source_v10_back_v1.txt iso_p7rep_source_v10_front_v1.txt iso_p7rep_source_v10_v1.txt
- 3/7/2013 - Run off 10 test runs (starting at beginning of mission with task version 1.0)
- 5/30/2013 - Green light given on new IRFs and diffuse model.
- Changes to config.py:
- New diffuse model: /afs/slac/g/glast/ground/GLAST_EXT/diffuseModels/v4r0
- New IRFs: P7REP_SOURCE_V15,P7REP_CLEAN_V15
- Awaiting a new ScienceTools release...
- Changes to config.py:
- 6/10/2013 - Several ST updates, and renaming of the diffuse model files. All appears good to go.
- 6/11/2013 - final tweaks to diffuse model v4r0 files; bump task version 1.0 -> 1.1 (delete version 1.0); and start...
- 6/14/2013 - block 1 complete (26,263 runs: 239557414 through 389089696; MET 2008-08-04 15:43:33 through 2013-05-01 08:28:13)
Name | Files | Events | Size |
---|---|---|---|
CAL | 26263 | 57,227,320,767 | 166.0 TB |
DIGIGAP | 24200 | 0 | 19.0 kB |
ELECTRONFT1 | 26263 | 0 | 10.9 GB |
ELECTRONMERIT | 26263 | 116,799,950 | 263.6 GB |
EXTENDEDFT1 | 26263 | 8,284,002,713 | 725.9 GB |
EXTENDEDLS1 | 26263 | 8,284,002,713 | 1.3 TB |
FILTEREDMERIT | 26263 | 8,284,037,323 | 7.0 TB |
FT1 | 26263 | 268,810,274 | 24.2 GB |
GCR | 26263 | 57,227,320,767 | 1.2 TB |
LS1 | 26263 | 1,782,493,106 | 289.6 GB |
MERIT | 26263 | 57,227,320,767 | 45.8 TB |
RECON | 26263 | 57,227,320,767 | 763.1 TB |
Discrepancy between FILTEREDMERIT and EXTENDED{LS1,FT1}. This turns out to be an issue with tstart/tstop for run 383219654.
R__unzip: error in header Error in <TBasket::ReadBasketBuffers>: fNbytes = 15550, fKeylen = 86, fObjlen = 31912, noutot = 0, nout=0, nin=11010118, nbuf=7864444 Error in <TBranch::GetBasket>: File: /scratch/glastmp/P202-FITS/3345/r0259101994_v202_merit.root at byte:459100837, branch:Vtx2LongDoca, entry:510338, badread=0, nerrors=1, basketnumber=1 [...]
Stream
Run
3345
259101994
<- found by FSSC
4122
263571912
4707
266893978
13927
319436826
16181
332306548
<- found by FSSC
17430
339161346
17479
339408141
- 10/15/2012 - Reconfigure for block 4 backfill and begin trickleStream.
- Discover run 22240 makeFT1 contains error messages as the mentioned last week (above). Rolling back the entire chain of processing, starting with P202-ROOT seemed to do the trick.
- 10/17/2012 - block4 backfill complete.
12/18/2012 - Prepare for block 5. Before:
Name
Type
Files
Events
Size
EXTENDEDFT1
Group
23141
7,252,996,601
662.6 GB
EXTENDEDLS1
Group
23141
7,252,996,601
1.1 TB
FT1
Group
23141
229,126,932
21.5 GB
LS1
Group
23141
1,549,133,207
251.7 GB
12/19/2012 - block5 backfill complete. Final DataCatalog numbers:
Name
Type
Files
Events
Size
EXTENDEDFT1
Group
24142
7,578,468,156
692.3 GB
EXTENDEDLS1
Group
24142
7,578,468,156
1.2 TB
FT1
Group
24142
240,949,332
22.6 GB
LS1
Group
24142
1,621,773,636
263.5 GB
- 3/6/2013 - Many changes!
- New task created (version 0.9 -> 1.0)
- Update to run native on RHEL5-64 and RHEL6-64
- New Interstellar Emission Model, v2r0 -> v3r0, provided by Luigi Tibaldo.
- Update ScienceTools from 09-27-01 to 09-31-01
- Update IRFS from P7*V6 to P7REP_V10 ( = CLEAN or SOURCE)
- This means that rather than rolling-back, the task starts from scratch.
- New evtClassDiffs version: 00-19-05, which changes P7V6 -> P7REP, the new designation.
Reference list of all existing FITS files (with old diffuse model) generated and stored here: /nfs/farm/g/glast/u38/Reprocess-tasks/P202-FITS/config/task-v0.9
Code Block diffmodel_p7rep_clean_v10_reduced.xml diffmodel_p7rep_clean_v10.xml diffmodel_p7rep_source_v10_reduced.xml diffmodel_p7rep_source_v10.xml gal_p7rep_v10_v1.fits gal_p7rep_v10_v1_reduced.fits iso_p7rep_clean_v10_back_v1.txt iso_p7rep_clean_v10_front_v1.txt iso_p7rep_clean_v10_v1.txt iso_p7rep_source_v10_back_v1.txt iso_p7rep_source_v10_front_v1.txt iso_p7rep_source_v10_v1.txt
- 3/7/2013 - Run off 10 test runs (starting at beginning of mission with task version 1.0)
- 5/30/2013 - Green light given on new IRFs and diffuse model.
- Changes to config.py:
- New diffuse model: /afs/slac/g/glast/ground/GLAST_EXT/diffuseModels/v4r0
- New IRFs: P7REP_SOURCE_V15,P7REP_CLEAN_V15
- Awaiting a new ScienceTools release...
- Changes to config.py:
- 6/10/2013 - Several ST updates, and renaming of the diffuse model files. All appears good to go.
- 6/11/2013 - final tweaks to diffuse model v4r0 files; bump task version 1.0 -> 1.1 (delete version 1.0); and start...
- 6/14/2013 - block 1 complete (26,263 runs: 239557414 through 389089696; MET 2008-08-04 15:43:33 through 2013-05-01 08:28:13)
Name | Files | Events | Size |
---|---|---|---|
CAL | 26263 | 57,227,320,767 | 166.0 TB |
DIGIGAP | 24200 | 0 | 19.0 kB |
ELECTRONFT1 | 26263 | 0 | 10.9 GB |
ELECTRONMERIT | 26263 | 116,799,950 | 263.6 GB |
EXTENDEDFT1 | 26263 | 8,284,002,713 | 725.9 GB |
EXTENDEDLS1 | 26263 | 8,284,002,713 | 1.3 TB |
FILTEREDMERIT | 26263 | 8,284,037,323 | 7.0 TB |
FT1 | 26263 | 268,810,274 | 24.2 GB |
GCR | 26263 | 57,227,320,767 | 1.2 TB |
LS1 | 26263 | 1,782,493,106 | 289.6 GB |
MERIT | 26263 | 57,227,320,767 | 45.8 TB |
RECON | 26263 | 57,227,320,767 | 763.1 TB |
Discrepancy between FILTEREDMERIT and EXTENDED{LS1,FT1}. This turns out to be an issue with tstart/tstop for run 383219654.
- 8/16/2013 - configure block2 backfill: 1610 new runs to reprocess
- 27873 total runs
- 60,682,674,790 total events
- Start run: 239557417 2008-08-04 15:43:37
- Last run: 398086126 2013-08-13 11:28:46
- 8/21/2013 - block2 complete (after redoing the entire block due to bizarre effects of last Friday's fermi-xrd005 crash). DataCatalog contents as of this morning:
Name | Files | Events | Size | Created (UTC) |
---|---|---|---|---|
CAL | 27873 | 60,682,674,790 | 175.8 TB | 25-Jan-2012 00:53:31 |
ELECTRONFT1 | 27873 | 0 | 11.5 GB | 02-Mar-2012 00:06:07 |
ELECTRONMERIT | 27873 | 123,494,286 | 278.5 GB | 25-Jan-2012 00:53:32 |
EXTENDEDFT1 | 27873 | 8,811,129,094 | 772.1 GB | 02-Mar-2012 00:06:09 |
EXTENDEDLS1 | 27873 | 8,811,129,094 | 1.4 TB | 02-Mar-2012 00:06:09 |
FILTEREDMERIT | 27873 | 8,811,129,090 | 7.5 TB | 25-Jan-2012 00:53:29 |
FT1 | 27873 | 289,969,364 | 26.1 GB | 02-Mar-2012 00:06:06 |
GCR | 27873 | 60,682,674,790 | 1.3 TB | 25-Jan-2012 00:53:31 |
LS1 | 27873 | 1,903,568,484 | 309.3 GB | 02-Mar-2012 00:06:08 |
MERIT | 27873 | 60,682,674,790 | 48.6 TB | 25-Jan-2012 00:53:30 |
RECON | 27873 | 60,682,674,790 | 808.5 TB | 25-Jan-2012 00:53:33 |
- 9/19/2013 - prepare block3 (corresponding to P202-ROOT block11).
- 9/20/2013 - block3 complete. One run from block2 was rolledback due to not using the latest MERIT file (run 383219654). Statistics are now consistent.
- 10/7/2013 - prepare block4 (corresponding to P202-ROOT block12).
10/8/2013 - block 4 complete
- 8/16/2013 - configure block2 backfill: 1610 new runs to reprocess
- 27873 total runs
- 60,682,674,790 total events
- Start run: 239557417 2008-08-04 15:43:37
- Last run: 398086126 2013-08-13 11:28:46
- 8/21/2013 - block2 complete (after redoing the entire block due to bizarre effects of last Friday's fermi-xrd005 crash). DataCatalog contents as of this morning:
Name
Files
Events
Size
Created (UTC)
...
ELECTRONMERIT
...
28675
...
126,
...
895,
...
805
...
286.
...
1 GB
25-Jan-2012 00:53:
...
32
...
FT1
...
27873
...
0
...
11.5 GB
...
02-Mar-2012 00:06:07
...
ELECTRONMERIT
...
27873
...
123,494,286
...
278.5 GB
28675
301,166,782
27.1 GB
02-Mar
...
-2012 00:
...
06:
...
06
...
LS1
...
28675
...
1,
...
964,
...
328,
...
038
...
319.1 GB
02-Mar-2012 00:06:
...
08
EXTENDEDLS1
...
28675
...
9,
...
072,
...
793,
...
026
1.4 TB
02-Mar-2012 00:06:09
FILTEREDMERIT
...
28675
...
9,
...
072,
...
793,
...
026
7.
...
7 TB
25-Jan-2012 00:53:29
...
EXTENDEDFT1
...
28675
...
9,072,
...
793,
...
026
...
795.
...
0 GB
02-Mar-2012 00:06:
...
09
...
MERIT
...
28675
...
62,
...
383,
...
754,
...
1.3 TB
...
25-Jan-2012 00:53:31
997
49.9 TB
25-Jan
...
LS1
...
27873
...
1,903,568,484
...
309.3 GB
...
-2012 00:
...
53:
...
30
...
CAL
...
28675
...
62,
...
383,
...
754,
...
997
...
180.6 TB
25-Jan-2012 00:53:
...
31
...
GCR
...
28675
...
62,
...
383,
...
754,
...
997
...
1.
...
3 TB
25-Jan-2012 00:53:
...
31
RECON
28675
62,383,754,997
831.0 TB
25-Jan-2012 00:53:33
- 10/17/2013 - prepare block5 backfill (P202-ROOT block13
- 9/19/2013 - prepare block3 (corresponding to P202-ROOT block11).
- 9/20/2013 - block3 complete. One run from block2 was rolledback due to not using the latest MERIT file (run 383219654). Statistics are now consistent.
- 10/7/2013 - prepare block4 (corresponding to P202-ROOT block12).
10/824/2013 - block 4 complete5 complete (as of 10/18/2013)
Name
Files
Events
SizeCreated (UTC)
ELECTRONMERIT
2867528844
126127,895613,805745
286287.1 GB25-Jan-2012 00:53:327 GB
FT1
2867528844
301303,166135,782818
27.1 3 GB
02-Mar-2012 00:06:06
LS1
2867528844
1,964976,328418,038512
319321.1 GB02-Mar-2012 00:06:08
EXTENDEDLS1
2867528844
9,072127,793005,026732
1.4 TB
02-Mar-2012 00:06:09
FILTEREDMERIT
2867528844
9,072127,793005,026732
7.7 TB
25-Jan-2012 00:53:29
EXTENDEDFT1
2867528844
9,072127,793005,026732
795799.0 GB02-Mar-2012 00:06:098 GB
MERIT
2867528844
62,383747,754048,997217
4950.9 2 TB
25-Jan-2012 00:53:30
CAL
2867528844
62,383747,754048,997217
180181.6 TB
25-Jan-2012 00:53:31
GCR
2867528844
62,383747,754048,997217
1.3 TB
25-Jan-2012 00:53:31
RECON
2867528844
62,383747,754048,997217
831835.0 8 TB
25-Jan-2012 00:53:33
- 10/17/2013 - prepare block5 backfill (P202-ROOT block13).
11/7/2013 - block 6 complete and P7REP was made public two days ago. END OF PROJECT
Final tally by the dataCatalog:Name Files Events Size Created (UTC) CAL 29158 63,449,261,789 183.6 TB 25-Jan-2012 00:53:31 ELECTRONMERIT 29158 128,971,706 290.7 GB 25-Jan-2012 00:53:32 EXTENDEDFT1 29158 9,243,063,549 809.9 GB 02-Mar-2012 00:06:09 EXTENDEDLS1 29158 9,243,063,549 1.5 TB 02-Mar-2012 00:06:09 FILTEREDMERIT 29158 9,243,063,549 7.8 TB 25-Jan-2012 00:53:29 FT1 29158 310,326,817 27.9 GB 02-Mar-2012 00:06:06 GCR 29158 63,449,261,789 1.3 TB 25-Jan-2012 00:53:31 LS1 29158 2,007,123,864 326.1 GB 02-Mar-2012 00:06:08 MERIT 29158 63,449,261,789 50.8 TB 25-Jan-2012 00:53:30 RECON 29158 63,449,261,789 845.0 TB 25-Jan-2012 00:53:33 8/20/2014 - Using new MERIT file from P202-ROOT (see log entry above), update fullList.txt and then rollback/regenerate FITS files for run 395891323, stream 27487.
10/24/2013 - block 5 complete (as of 10/18/2013)
Name
Files
Events
Size
ELECTRONMERIT
28844
127,613,745
287.7 GB
FT1
28844
303,135,818
27.3 GB
LS1
28844
1,976,418,512
321.1 GB
EXTENDEDLS1
28844
9,127,005,732
1.4 TB
FILTEREDMERIT
28844
9,127,005,732
7.7 TB
EXTENDEDFT1
28844
9,127,005,732
799.8 GB
MERIT
28844
62,747,048,217
50.2 TB
CAL
28844
62,747,048,217
181.6 TB
GCR
28844
62,747,048,217
1.3 TB
RECON
28844
835.8 TB62,747,048,217
Configuration
Task Location | /nfs/farm/g/glast/u38/Reprocess-tasks/P202-FITS |
Task Status | http://glast-ground.slac.stanford.edu/Pipeline-II/task.jsp?task=107152539 |
Input Data | MERIT (direct from P202-ROOT) |
spacecraft data | same as P202-ROOT |
Input Run List | ftp://ftp-glast.slac.stanford.edu/glast.u38/Reprocess-tasks/P202-FITS/config/runList.txt |
evtClassDefs | 00-19-05 (March 2013, changed pass_ver to P7REP) |
eventClassMap | EvtClassDefs_P7V6.xml |
ScienceTools | 09-32-03 (6/7/2013) (but ST may report themselves as 09-32-02 due to RM snafu) |
Code Variants | redhat5-x86_64-64bit-gcc41, redhat6-x86_64-64bit-gcc44 (Optimized) |
Diffuse Model | based on contents of /afs/slac.stanford.edu/g/glast/ground/GLAST_EXT/diffuseModels/v4r0 |
Diffuse Response | 'source' using P7REP_SOURCE_V15 IRF |
IRFs | P7REP_*_V15, contained within ScienceTools release |
Output Data Products |
...