Page History

Versions Compared

Key

This line was added.
This line was removed.
Formatting was changed.

...

top-level stream	sub-stream	type	primary function
setupRun		py	discover run number, input files; calculate # parallel substreams
createClumps		jy	create substreams
	processClump	py	Gleam reprocessing of run fragment (clump)
	clumpDone	jy	no-op
setupMerge		jy	collect data from Pipeline II DB and write to file
mergeClumps		py	merge output from processClump; create post-merge data products
runDone		jy	register datasets; update processing history DB

Task Details

setupRun.py

read & parse 'runList' file
identify run# and input files for this stream
calculate #clumps (substreams in subtask)
create env-vars for subtask

...

unpack pipeline-vars (as env-vars)
define skipEvents/lastEvent for Gleam
Wiki Markup
\[limit # events processed\]
Wiki Markup
infile staging \[disabled\]
Wiki Markup
infile slicing (skimmer) \[disabled\]
infile env-vars for Gleam
construct output filenames
output file staging
set output file env-vars for Gleam
prepare .rootrc
select and stage-in FT2 file
- (Gleam setup)
select jobOptions file and customize
Run Gleam
- (SVAC setup)
Wiki Markup
Run svac \[disabled\]
Wiki Markup
Run makeFT1 \[disabled\]
Finalize staging
create new subTask-level pipeline-vars with clump output info

...