You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 10 Next »

P118 Reprocessing

status: In preparation
last update: 15 Feb 2011

This page is a record of the configuration for the P118 reprocessing project, recalculating diffuse response with new P6_V11 IRFs and adding to existing FT1 files. This processing will be applied to all science data, be incorporated into Level 1 processing, and results sent to the FSSC.

  • P118-FT1 - this task will read existing FT1 (photons) files and emit new FT1 files

Datafile names, versions and locations

Data file version numbers for this reprocessing will begin with v118.

XROOT location and file naming

Location template:

/glast/Data/Flight/Reprocess/<reprocessName>/<dataType>

Locations for P118:

/glast/Data/Flight/Reprocess/P118/ft1

File naming:

Data Type

aka

Send to FSSC

Naming template

FT1

LS-002

Yes

gll_ph_p<procVer>_r<run#>_<version>.fit

Note: 'procVer' is a field added to the file name (and the keyword "PROC_VER" in the primary header) added to the FFD 5/12/2010. Ref: http://fermi.gsfc.nasa.gov/ssc/dev/current_documents/Science_DP_ICD_RevA.pdf

Example:

/glast/Data/Flight/Reprocess/P118/ft1/gll_ph_p116_r0239559565_v116.fit
/glast/Data/Flight/Reprocess/P118/ls1/gll_ev_p116_r0239559565_v116.fit
DataCatalog location and naming

Logical directory and group template:

Data/Flight/Reprocess/<reprocessName>:<dataType>

Note that the <dataType> field (following the colon) is a DataCatalog 'group' name, and file names are of the form r<run#>.

Naming examples:

Data/Flight/Reprocess/P118:FT1 r0239557414
Data/Flight/Reprocess/P118:LS1 r0239557414

Data Sample

The data sample for P118 reprocessing includes the current definition of the "Public_v2" dataset.

Block

First run
MET

First run
UTC

Last run
MET

Last run
UTC

#Runs

#Evts

DataCatalog Source

Note

1

239557414

2008-08-04 15:43:34

311669624

2010-11-17 06:53:42

12557

435,741,248

/Data/Flight/Reprocess/P116/FT1

entire P116 dataset

1

311675447

2010-11-17 08:30:45

319652280

2011-02-17 16:17:58

1410

55,911,339

/Data/Flight/Level1/LPA/

Standard Level 1 output

2

 

 

 

 

 

 

 

 

Final number of selected photon events (in the FT1 and LS1 files) = TBD

These data (along with subsequent Level 1 data) will be in the SLAC Astroserver as P6_public_v3.




Bookkeeping

  1. (This page): Define ingredients of reprocessing (processing code/configuration changes)
  2. Processing History database: http://glast-ground.slac.stanford.edu/HistoryProcessing/HProcessingRuns.jsp?processingname=P118
    1. List of all reprocessings
    2. List of all data runs reprocessed
    3. Pointers to all input data files (-> dataCatalog)
    4. Pointers to associated task processes (-> Pipeline II status)
  3. Data Catalog database: http://glast-ground.slac.stanford.edu/DataCatalog/folder.jsp
    1. Lists of and pointers to all output data files
    2. Meta data associated with each output data product

P118-FT1

(This task is equivalent to the P116-FT1 task but with all but gtdiffrsp step bypassed.)

Status chronology

2/22/2011 - Begin trickleStream of block1
2/15/2011 - Begin configuration for this task

Configuration

Task Location

/nfs/farm/g/glast/u38/Reprocess-tasks/P118-FT1

Task Status

P118-FT1

Input Data Selection

FT1

Input Run List

http://glast-ground.slac.stanford.edu/Decorator/Decorate/u38/Reprocess-tasks/P118-FT1/config/runFile.txt

Reprocessing Mode

doDiffRsp

evtClassDefs

n/a (see P116)

photon cut

n/a (see P116)

eventClassifier

n/a (see P116)

s/c data

FT2 from P105 (runs 239557414 - 271844560), then from current Level 1 production

ScienceTools

09-21-00 (SCons build)

Allowed Code Variants

redhat4-i686-32bit-gcc34, redhat4-x86_64-64bit-gcc34,
redhat5-i686-32bit-gcc41, redhat5-x86_64-64bit-gcc41 (Optimized)

Diffuse Model

/afs/slac/g/glast/groups/diffuse/mapcubes/source_model_v02_P6_V11_DIFFUSE.xml
(

https://confluence.slac.stanford.edu/display/SCIGRPS/Diffuse+Model+for+Analysis+of+LAT+Data

)

Diffuse Response IRFs

P6_V11_DIFFUSE

Output Data Products

FT1

Processing chain for FITS data products

Data Product

makeFT1

gtdiffrsp

gtmktime

gtltcube

FT1

false

true for
evclsmin==3

false

false

Note on 'Code Variant': The SLAC batch farm contains a mixture of architectures , both hardware (Intel/AMD 64-bit) and software (RedHat Enterprise Linux 4, and 5, gcc 3.4, 4.1, etc.). Currently, all batch machines have 64-bit hardware and (most, if not all) run with the 64-bit operating systems.

Timing

The execution of gtdiffrsp happens in the mergeClumps step and takes ~5 hequ-minutes to complete.

  • No labels