launchChunks: is just an embedded jython scriptlet launching a stream for every chunk: for i in range(0, numChunks): pipeline.createSubstream("doChunk", i+1)
doChunk.makeEvt.sh
each makeEvt creates identical lock files ${HALFPIPE_OUTPUTBASE}/lock/halfpipe-${HALFPIPE_DOWNLINKID}. Have to check when this file is being removed.
RetDef files $HALFPIPE_OUTPUTBASE/$HALFPIPE_DOWNLINKID/RetDef-$HALFPIPE_DOWNLINKID-$HALFPIPE_CHUNKID.xml are missing. u42 isn't on s3df. We copied these files from rhel6-64:/nfs/farm/g/glast/u42/ISOC-flight/Downlinks/<dl>/.
mergeIndices.sh
tokendir is read dinamically: tokendir=`cat ${taskBase}/config/stagedir`/chunktokens. For testing we changed stagedir from /nfs/farm/g/glast/u28/stage to /sdf/group/fermi/a/isoc/flightOps/volumes/vol5/offline/halfPipe/v7r0p0_devel/u28/stage.
at the end a mergeEvt stream for each run is created. This won't work in the container. This task is being taken by launchMerge, an embedded scriptlet in the xml.
launchMerge: embedded scriptlet to launch mergeEvt
mergeEvt.sh
should receive ${HALFPIPE_RUNSTART} from launchMerge. Doesn't work right now. Workaround: read the decimal runid from PIPELINE_STREAM and convert to hex. Michael thinks this should be safe.
not fault of mergeEvt, I guess: mergeEvt runs always on sdfmilan11[56]. There is no /lscratch but /scratch, and the outputdir is owned by 7234, while glastraw is 9503. How to report this?
Michael did something else yesterday to make mergeEvt finish. But probably the above should be understood and fixed first.