P120 Reprocessing

status: Under construction
last update: 19 July 2010

This page is a record of the configuration for the P120 reprocessing project, event reclassification using Pass 7.3. This project involves reprocessing with Pass7 classification trees and (ultimately) new IRFs. The name "P120" derives from the word "processing" and the initial file version to be used for the output data products, e.g., r0123456789_v120_merit.root.

• P120-MERIT - this task reads DIGI+RECON+MERIT and produces reprocessed MERIT + FILTEREDMERIT (photons) + ELECTRONMERIT

• P120-FITS - this task will (eventually) read FILTEREDMERIT and produce FT1 (photons) + LS1 (merit-like FITS file for photons) + electron FITS file + LS3 (live-time cube)

Datafile names, versions and locations

Data file version numbers for this reprocessing will begin with v110.

XROOT location and file naming

Location template:

/glast/Data/Flight/Reprocess/<reprocessName>/<dataType>

Locations for P120:

/glast/Data/Flight/Reprocess/P120/merit
/glast/Data/Flight/Reprocess/P120/filteredmerit
/glast/Data/Flight/Reprocess/P120/electronmerit
/glast/Data/Flight/Reprocess/P120/ft1
/glast/Data/Flight/Reprocess/P120/electronft1
/glast/Data/Flight/Reprocess/P120/ls1
/glast/Data/Flight/Reprocess/P120/ls3

File naming:

Example:

/glast/Data/Flight/Reprocess/P120/merit/r0239557414_v120_merit.root
/glast/Data/Flight/Reprocess/P120/filteredmerit/r0239557414_v120_filteredmerit.root
/glast/Data/Flight/Reprocess/P120/electronmerit/r0239557414_v120_electronmerit.root
/glast/Data/Flight/Reprocess/P120/ft1/gll_ph_r0239559565_v120.fit
/glast/Data/Flight/Reprocess/P120/electronft1/r0239557414_v120_electronft1.fit
/glast/Data/Flight/Reprocess/P120/ls1/gll_ev_r0239559565_v120.fit
/glast/Data/Flight/Reprocess/P120/ls3/gll_lt_r0239559565_v120.fit

DataCatalog location and naming

Logical directory and group template:

Data/Flight/Reprocess/<reprocessName>:<dataType>

Note that the <dataType> field (following the colon) is a DataCatalog 'group' name, and file names are of the form r<run#>.

Naming examples:

Data/Flight/Reprocess/P120:MERIT r0239557414
Data/Flight/Reprocess/P120:FILTEREDMERIT r0239557414
Data/Flight/Reprocess/P120:FT1 r0239557414
Data/Flight/Reprocess/P120:LS1 r0239557414
Data/Flight/Reprocess/P120:LS3 r0239557414

Data Sample

The currently defined data sample for P120 reprocessing includes:

Bookkeeping

1. (This page): Define ingredients of reprocessing (processing code/configuration changes)
2. Processing History database: http://glast-ground.slac.stanford.edu/HistoryProcessing/HProcessingRuns.jsp?processingname=P120
   1. List of all reprocessings
   2. List of all data runs reprocessed
   3. Pointers to all input data files (-> dataCatalog)
   4. Pointers to associated task processes (-> Pipeline II status)
3. Data Catalog database: http://glast-ground.slac.stanford.edu/DataCatalog/folder.jsp
   1. Lists of and pointers to all output data files
   2. Meta data associated with each output data product

P120-MERIT

Status chronology

Configuration

WARNING: NOT UP TO DATE!

Task Location	/nfs/farm/g/glast/u38/Reprocess-tasks/P120-MERIT
Task Status	http://glast-ground.slac.stanford.edu/Pipeline-II/index.jsp
GlastRelease	v17r35p8
Input Data Selection	"standard" from https://confluence.slac.stanford.edu/display/SCIGRPS/LAT+Dataset+Definitions along with "&& (RunQuality != "Bad" || is_null ( RunQuality )"
Input Run List	ftp://ftp-glast.slac.stanford.edu/glast.u38/Reprocess-tasks/P120-MERIT/config/runFile.txt
photonFilter	evtClassDefs v0r6p1 CTBParticleType==0 && CTBClassLevel>0
electronFilter	CTBParticleType==1
jobOpts	ftp://ftp-glast.slac.stanford.edu/glast.u38/Reprocess-tasks/P120-MERIT/config/reClassify.txt
Output Data Products	MERIT, FILTEREDMERIT, ELECTRONMERIT

Timing

n/a

P120-FT1

Status chronology

Configuration

WARNING: NOT UP TO DATE!

Task Location	/nfs/farm/g/glast/u38/Reprocess-tasks/P120-FT1
Task Status	http://glast-ground.slac.stanford.edu/Pipeline-II/index.jsp
Input Data Selection	MERIT (from P120-MERIT), FT2 (from P100-FT2 and Level1)
Input Run List	ftp://ftp-glast.slac.stanford.edu/glast.u38/Reprocess-tasks/P120-FT1/config/runFile.txt
evtClassDefs	00-16-00
meritFilter	pass7_FSW_cuts, (FswGamState==0||FswGamState==3) && (CTBCORE>0) && (CTBBestEnergyProb>0) && (CTBBestEnergy>10) && (CTBBestEnergyRatio<5) && (CTBClassLevel>0)
eventClassifier	Pass7_Classifier.py
ScienceTools	09-15-05 (SCons build)
Code Variant	forced to redhat4-i686-32bit-gcc34
Diffuse Model	/afs/slac.stanford.edu/g/glast/ground/releases/analysisFiles/diffuse/v2/source_model_v02.xml ( https://confluence.slac.stanford.edu/display/SCIGRPS/Diffuse+Model+for+Analysis+of+LAT+Data )
Diffuse Response IRFs	P7_v2_diff, P7_v2_extrad, P7_v2_datac
IRFs	implemented as 'custom irf', files in /afs/slac.stanford.edu/g/glast/ground/PipelineConfig/IRFS/Pass7.2
Output Data Products	FT1, LS1, LS3, ELECTRONFT1

Processing chain for FITS data products

Note on 'Code Variant': The SLAC batch farm contains a mixture of architectures , both hardware (Intel/AMD and 32-/64-bit) and software (RedHat Enterprise Linux 3, 4 and 5, gcc 3.2, 3.4, 4.1, etc.). GLAST/Fermi code builds on many newer combinations, but is not yet validated on them.

Note on diffuse response calculation: gtdiffrsp is called three times in succession. The first time with IRF P7_v2_diff and evclsmin==8, followed by IRF P7_v2_extrad and evclsmin==9, and finally IRF P7_v2_datac and evclsmin==10. The resulting FT1 file has six columns of diffuse response, two columns (galactic and extragalactic response) for each of the three IRFs. This creates a non-standard FT1 file by FSSC standards as they expect only five diffuse response columns.

Timing

n/a