Versions Compared

Old Version 6

changes.mady.by.user Tom Glanzman

Saved on

compared with

New Version 7

changes.mady.by.user Tom Glanzman

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

status: In Development Running
last update: 10 May 2 July 2013

This page is a record of the configuration and execution of the P300 reprocessing project, full reprocessing from DIGIs using Pass8 analysis code.

...

  • P300-ROOT - This task reads DIGI and produces reprocessed RECON + GCR + MERIT + FILTEREDMERIT (photons) + ELECTRONMERIT
  • P300-FITS - This task (once it has been created) reads MERIT and produces FT1 (photons) + EXTENDEDFT1 + LS1 (merit-like FITS file for photons) + EXTENDEDLS1 + ELECTRONFITS file

...

The currently defined data sample (as of XXXXXX7/2/2013) for P300 reprocessing includes:

First run

239557414 (MET), 2008-08-04 15:43:34 (UTC)

Last run

354923690 (MET), 2012-03-31 21:54:48 (UTC) present

Bookkeeping

  1. (This page): Define ingredients of reprocessing (processing code/configuration changes)
  2. Processing History database: http://glast-ground.slac.stanford.edu/HistoryProcessing/HProcessingRuns.jsp?processingname=P300
    1. List of all reprocessings
    2. List of all data runs reprocessed
    3. Pointers to all input data files (-> dataCatalog)
    4. Pointers to associated task processes (-> Pipeline II status)
  3. Data Catalog database: http://glast-ground.slac.stanford.edu/DataCatalog/folder.jsp
    1. Lists of and pointers to all output data files
    2. Meta data associated with each output data product

...

  • 4/23/2013 - Begin setting up P300 task for Pass8 reprocessing.
  • 5/10/2013 - Initial version of task running.  See this page for infomation about the task creation, and this page for performance comparison with P202
  • 7/2/2013 - after weeks of testing, final GR, jobOpts, etc. come together and production tentatively starts.
    • block 1 is defined with this run range: ?239557414 through 392842073 (2008-08-04 15:43:37 through 2013-06-13 18:47:50 UTC)

Configuration

NOT YET UPDATED

Task Location

/nfs/farm/g/glast/u38/Reprocess-tasks/P300-ROOT

Task Status

http://glast-ground.slac.stanford.edu/Pipeline-II/index.jsp

GlastRelease

20-08-12 (SCons RHEL5-32 build) 09-00

Run Selection

based on a modified "standard" selection, see https://confluence.slac.stanford.edu/display/SCIGRPS/Official+LAT+Datasets
(((sIntent=="nomSciOps" || sIntent=="nomSO_noSk_noCno_optGccc_allEna" || sIntent=="nomSciOps_diagEna" || (sIntent=="nomSciOps_Emin5MeV"&&RunMin>242070455) || nRun==242429468 ) && (RunQuality != "Bad" || is_null ( RunQuality ) ) ) || sIntent=="nadirOps" )

s/c data

P202 FT2SECONDS which will eventually become a "standard" Public Release 2 Release https://confluence.slac.stanford.edu/display/SCIGRPS/Official+LAT+Datasets

Input Run List

ftp://ftp-glast.slac.stanford.edu/glast.u38/Reprocess-tasks/P300-ROOT/config/runList.txt

photonFilter

CTBParticleType Not yet applicable, although this is defined by default:  CTBParticleType==1 && ((FT1EventClass & 0x00003EFF)!=0)
pass7.6_Extended_cuts_L1 in evtClassDefs

electronFilter

CTBParticleType Not yet applicable, although this is defined by default:  CTBParticleType==1

Code Variants

redhat5-i686-32bit-gcc41 (Optimized), note that rhel5-64 and rhel6-64 GR builds are not yet available

jobOpts

ftp://ftp-glast.slac.stanford.edu/glast.u38/Reprocess-tasks/P300-ROOT/config/Pass8Recon.txt

Output Data Products

RECON, GCR, RELATION, MERIT, FILTEREDMERIT, ELECTRONMERIT (but note that FILTEREDMERIT and ELECTRONMERIT are empty or contain junk)

Timing and Scaling

  • Current performance results for the Pass8 code are on this page .
  • ||job step||average CPU time||
    |processClump|183 min|
    |mergeClumps|10 min|
  •  No information about scaling limits on the batch farm is currently known.

P300-FITS
Anchor
P300-FITS
P300-FITS

This task generates all desired FITS data products. This task does not yet exist.

Status chronology

Configuration

...

<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="325d9842f05e2c0b-3284bac4-4b9b4a89-9177bde3-3803a2f5fe1357d451e5c79a"><ac:plain-text-body><![CDATA[

Data Product

destination

data content [1]

event selection [1]

makeFT1

gtselect

gtdiffrsp

gtmktime

]]></ac:plain-text-body></ac:structured-macro>

EXTENDEDFT1

SLAC

FT1variables

((FT1EventClass & 0x00003EFF)!=0)
pass7.6_Extended_cuts_L1

(tick)

(error)

(tick)

(tick)

FT1

FSSC+SLAC

FT1variables

'source' and above
EVENT_CLASS bits 2,3,4
evclass=2 filtered from EXTENDEDFT1

(error)

(tick)

(inherited)

(tick)

EXTENDEDLS1

SLAC

LS1variables

((FT1EventClass & 0x00003EFF)!=0)
pass7.6_Extended_cuts_L1

(tick)

(error)

(tick)

(tick)

LS1

FSSC+SLAC

LS1variables

'transient' and above
EVENT_CLASS bits 0,2,3,4
evclass=0 filtered from EXTENDEDLS1

(error)

(tick)

(inherited)

(tick)

ELECTRONFT1

SLAC

FT1variables

CTBParticleType==1
pass7.6_Electrons_cuts_L1

(tick)

(error)

(error)

(tick)

...