You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 4 Next »

Reprocessing task structure

1/11/2012 Under Construction

Directories

/nfs/farm/g/glast/u38/Reprocess-tasks

commonTools - common file available to all tasks
P201-ROOT - specific task producing (primarily) ROOT files, e.g., runs Gleam
P201-FITS - specific task producing (primarily) FITS files, e.g., creates FT1, LS1, etc.

commonTools

/nfs/farm/g/glast/u38/Reprocess-tasks/commonTools/00-01-00

findRunsRepro.py*

search dataCatalog (used to generate run list)

checkRunList.py*

check/summarize run list(s)

 

 

envCtl.py*

class for env-var management

svs.py*

SCons variant string generator

pickleEnv.py*

wrapper for python setup scripts (pickles resultant env)

pickleEnv.sh*

wrapper for bash setup scripts (pickles resultant env)

 

 

repTools.py

miscellaneous functions used by repro tasks

 

makeXrootFilename()

 

makeXrootScratchFilename()

 

getKey()

 

findFt2()

 

getFile()

 

getCurrentVersion()

 

fileOpen()

 

runMakeFT1()

 

 

ft1skim.py*

wrapper to run ft1skimmer (TonyJ) (OBSOLETE?)

runFT1skim.sh

wrapper to ft1skim.py (OBSOLETE?)

 

 

runMakeFT1.sh*

wrapper to run makeft1/gtdiffrsp/gtmktime

runSvac.sh*

wrapper to run SVAC tuple generation code

runWrapper.sh*

(old) wrapper to run Gleam (CMT version)

 

 

setupFtools.sh*

create environment to run FTools

setupGR.sh*

create environment to run Gleam (GR)

setupOracle.sh*

create environment for Oracle

setupSkimmer.py

create environment to run TSkim

setupST.sh*

create environment to run ScienceTools

setupXroot.sh*

create environment to run xroot tools

 

 

trickle.py

class to control rate of task stream creation

 

 

.xrayrc

FTools setup config file

task-specific code

Task preparation

 

taskConfig.xml

task definition

genRunFile.csh*

generate list of input files for reprocessing

 

 

Pipeline code

 

envSetup.sh*

set up environment to run GR/ST/FT/etc (called by pipeline)

config.py

task configuration (imported by all .py)

setupRun.py*

setup for reprocessing a single run

createClumps.jy

create subprocess for processing a "clump" (part of a run)

processClump.py*

process a clump of data

clumpDone.jy

cleanup after clump processing

setupMerge.jy

setup for merging clumps

mergeClumps.py*

merge all clumps for single run

runFT1skim.sh*

skim FT1 events

runDone.jy

final bookkeeping after run reprocessed (dataCat and runHistory)

commonTools@

link to commonTools

 

 

Input data to pipeline code

 

doRecon.txt

Gleam job options

fullList.txt

List of reprocessing input data files

removeMeritColumns.txt

List of columns to remove from MERIT files

runFile.txt@

Sym link to fullList.txt

 

 

Pipeline control code

 

trickleStream.py*

task-specific config for trickle.py

  • No labels