Page History

...

• launchChunks: is just an embedded jython scriptlet launching a stream for every chunk: for i in range(0, numChunks): pipeline.createSubstream("doChunk", i+1)
• doChunk.makeEvt.sh
  • each makeEvt creates identical lock files ${HALFPIPE_OUTPUTBASE}/lock/halfpipe-${HALFPIPE_DOWNLINKID} .  Have to check when this file is being removed.
  • RetDef files $HALFPIPE_OUTPUTBASE/$HALFPIPE_DOWNLINKID/RetDef-$HALFPIPE_DOWNLINKID-$HALFPIPE_CHUNKID.xml are missing.  u42 isn't on s3df.  We copied these files from rhel6-64:/nfs/farm/g/glast/u42/ISOC-flight/Downlinks/<dl>/ .
• mergeIndices.sh
  • tokendir is read dinamically: tokendir=`cat ${taskBase}/config/stagedir`/chunktokens . For testing we changed stagedir from /nfs/farm/g/glast/u28/stage to /sdf/group/fermi/a/isoc/flightOps/volumes/vol5/offline/halfPipe/v7r0p0_devel/u28/stage .
  • at the end a mergeEvt stream for each run is created.  This won't work in the container.  This task is being taken by launchMerge , an embedded scriptlet in the xml.
• launchMerge: embedded scriptlet to launch mergeEvt
• mergeEvt.sh
  • should receive ${HALFPIPE_RUNSTART} from launchMerge.  Doesn't work right now.  Workaround: read the decimal runid from PIPELINE_STREAM and convert to hex.  Michael thinks this should be safe.
  • not fault of mergeEvt, I guess: mergeEvt runs always on sdfmilan11[56].  There is no /lscratch but /scratch /lscratch needed --bind and /sdf/group/fermi/a/isoc/flightOps/volumes/vol5/offline/halfPipe/v7r0p0_devel/outputBase is mapped to uid 7234 , while glastraw is 9503 .  Ticket open. Michael's fault.  7234 is Omodei, glastraw is 9503.  All ok.
  • Michael did something else yesterday to make mergeEvt finish.  But probably the above should be understood and fixed first.
• launchL1
  • should work (I (Nicola) haven’t added the "submit" script yet, but it gets to the point where it call the “submit stream” command successfully created.
  • launchL1 creates streams of task intOnlineAnalysis.  It seems to deal with calibration runs.  The last time an intOnlineAnalysis stream finished successfully was 2013, the last partial 2017!  We have to figure out if this task has any use, needs to be fixed, or simply can be skipped.
• launchOnline: there is still the question about the “evt” directory… we need to check on the rhel6 who is making this dir and how it is used? We also have multiple directories called “stage” (one in  /sdf/group/fermi/a/isoc/flightOps/volumes/vol5/offline/halfPipe/v7r0p0_devel/u28/stage/  one in  /sdf/group/fermi/a/isoc/flightOps/volumes/vol5/offline/halfPipe/v7r0p0_devel/outputBase/stage/  we probably need a bit of cleaning.