P118 Reprocessing
status: In preparation
last update: 15 Feb 2011
This page is a record of the configuration for the P118 reprocessing project, recalculating diffuse response with new P6_V11 IRFs and adding to existing FT1 files. This processing will be applied to all science data, be incorporated into Level 1 processing, and results sent to the FSSC.
Datafile names, versions and locations
Data file version numbers for this reprocessing will begin with v118.
XROOT location and file naming
Location template:
/glast/Data/Flight/Reprocess/<reprocessName>/<dataType>
Locations for P118:
/glast/Data/Flight/Reprocess/P118/ft1
File naming:
Data Type |
aka |
Send to FSSC |
Naming template |
---|---|---|---|
FT1 |
LS-002 |
Yes |
gll_ph_p<procVer>_r<run#>_<version>.fit |
Note: 'procVer' is a field added to the file name (and the keyword "PROC_VER" in the primary header) added to the FFD 5/12/2010. Ref: http://fermi.gsfc.nasa.gov/ssc/dev/current_documents/Science_DP_ICD_RevA.pdf
Example:
/glast/Data/Flight/Reprocess/P118/ft1/gll_ph_p116_r0239559565_v116.fit /glast/Data/Flight/Reprocess/P118/ls1/gll_ev_p116_r0239559565_v116.fit
DataCatalog location and naming
Logical directory and group template:
Data/Flight/Reprocess/<reprocessName>:<dataType>
Note that the <dataType> field (following the colon) is a DataCatalog 'group' name, and file names are of the form r<run#>.
Naming examples:
Data/Flight/Reprocess/P118:FT1 r0239557414 Data/Flight/Reprocess/P118:LS1 r0239557414
Data Sample
The data sample for P118 reprocessing includes the current definition of the "Public_v2" dataset.
Block |
First run |
First run |
Last run |
Last run |
#Runs |
#Evts |
DataCatalog Source |
Note |
---|---|---|---|---|---|---|---|---|
1 |
239557414 |
2008-08-04 15:43:34 |
311669624 |
2010-11-17 06:53:42 |
12559 |
435,801,179 |
/Data/Flight/Reprocess/P116/FT1 |
entire P116 dataset |
1 |
311675447 |
2010-11-17 08:30:45 |
(present) TBD |
TBD |
|
|
/Data/Flight/Level1/LPA/ |
Standard Level 1 output |
Final number of selected photon events (in the FT1 and LS1 files) = TBD
These data (along with subsequent Level 1 data) will be in the SLAC Astroserver as P6_public_v3.
Three extra runs were unintentionally processed as part of P116 what do do for P118??:
Run |
Task Stream |
Trigger |
LAT configuration flag |
Disposition |
---|---|---|---|---|
242429468 |
499 |
nomSciOps_trigTest |
LAT_CONFIG = 1 |
retained |
250687192 |
1948 |
hldVetoCalib_Hi |
LAT_CONFIG = 0 |
removed |
250692922 |
1949 |
hldVetoCalib_Lo |
LAT_CONFIG = 0 |
removed |
The first of these was deemed good for science, while the last two were not. Therefore, runs 250687192 and 250692922 have been retroactively removed from the datacatlog, astroserver and FSSC.
Bookkeeping
- (This page): Define ingredients of reprocessing (processing code/configuration changes)
- Processing History database: http://glast-ground.slac.stanford.edu/HistoryProcessing/HProcessingRuns.jsp?processingname=P118
- List of all reprocessings
- List of all data runs reprocessed
- Pointers to all input data files (-> dataCatalog)
- Pointers to associated task processes (-> Pipeline II status)
- Data Catalog database: http://glast-ground.slac.stanford.edu/DataCatalog/folder.jsp
- Lists of and pointers to all output data files
- Meta data associated with each output data product
P118-FT1
(This task is equivalent to the P116-FT1 task but with all but gtdiffrsp step bypassed.)
Status chronology
2/15/2011 - Begin configuration for this task
Configuration
Task Location |
/nfs/farm/g/glast/u38/Reprocess-tasks/P118-FT1 |
Task Status |
|
Input Data Selection |
FT1 |
Input Run List |
http://glast-ground.slac.stanford.edu/Decorator/Decorate/u38/Reprocess-tasks/P118-FT1/config/runFile.txt |
Reprocessing Mode |
doDiffRsp |
evtClassDefs |
n/a (see P116) |
photon cut |
n/a (see P116) |
eventClassifier |
n/a (see P116) |
s/c data |
FT2 from P105 (runs 239557414 - 271844560), then from current Level 1 production |
ScienceTools |
09-21-00 (SCons build) |
Allowed Code Variants |
redhat4-i686-32bit-gcc34, redhat4-x86_64-64bit-gcc34, |
Diffuse Model |
/afs/slac/g/glast/groups/diffuse/mapcubes/source_model_v02_P6_V11_DIFFUSE.xml ) |
Diffuse Response IRFs |
P6_V11_DIFFUSE |
Output Data Products |
Processing chain for FITS data products
Data Product |
makeFT1 |
gtdiffrsp |
gtmktime |
gtltcube |
---|---|---|---|---|
FT1 |
false |
true for |
false |
false |
Note on 'Code Variant': The SLAC batch farm contains a mixture of architectures , both hardware (Intel/AMD 64-bit) and software (RedHat Enterprise Linux 4, and 5, gcc 3.4, 4.1, etc.). Currently, all batch machines have 64-bit hardware and (most, if not all) run with the 64-bit operating systems.
Timing
(nothing yet)