- launchChunks: is just an embedded jython scriptlet launching a stream for every chunk:
for i in range(0, numChunks): pipeline.createSubstream("doChunk", i+1)
- doChunk.makeEvt.sh
- each makeEvt creates identical lock files
${HALFPIPE_OUTPUTBASE}/lock/halfpipe-${HALFPIPE_DOWNLINKID}
. Have to check when this file is being removed. - RetDef files
$HALFPIPE_OUTPUTBASE/$HALFPIPE_DOWNLINKID/RetDef-$HALFPIPE_DOWNLINKID-$HALFPIPE_CHUNKID.xml
are missing. u42 isn't on s3df. We copied these files from rhel6-64:/nfs/farm/g/glast/u42/ISOC-flight/Downlinks/<dl>/
.
- mergeIndices.sh
- tokendir is read dinamically:
tokendir=`cat ${taskBase}/config/stagedir`/chunktokens
. For testing we changed stagedir
from /nfs/farm/g/glast/u28/stage
to /sdf/group/fermi/a/isoc/flightOps/volumes/vol5/offline/halfPipe/v7r0p0_devel/u28/stage
. - at the end a mergeEvt stream for each run is created. This won't work in the container. This task is being taken by
launchMerge
, an embedded scriptlet in the xml.
- launchMerge: embedded scriptlet to launch mergeEvt
- mergeEvt.sh
- should receive ${HALFPIPE_RUNSTART} from launchMerge. Doesn't work right now. Workaround: read the decimal runid from PIPELINE_STREAM and convert to hex. Michael thinks this should be safe.
- not fault of mergeEvt, I guess: mergeEvt runs always on sdfmilan11[56]. There is no
/lscratch
but /scratch
, and the outputdir is owned by /sdf/group/fermi/a/isoc/flightOps/volumes/vol5/offline/halfPipe/v7r0p0_devel/outputBase
is mapped to uid 7234
, while glastraw is 9503
. How to report this? - Michael did something else yesterday to make mergeEvt finish. But probably the above should be understood and fixed first.
{"serverDuration": 42, "requestCorrelationId": "fd50053c034a8143"}