Page History

Incoming: On rh6 incoming raw data are in /nfs/farm/g/glast/u23/ISOC-flight/Archive/level0 (fastcopy in ../fcopy). End of April 2023 this dir was copied to /sdf/group/fermi/n/u23/ISOC-flight/Archive/level0. That's why tests on dev are done with runs of April1, 2023. Additionally we need RetDef files, which are in rhel6-64:/nfs/farm/g/glast/u42/ISOC-flight/Downlinks/<dl>/. Which process creates these files?

• launchChunks: is just an embedded jython scriptlet launching a stream for every chunk: for i in range(0, numChunks): pipeline.createSubstream("doChunk", i+1)
• doChunk.makeEvt.sh
  • each makeEvt creates identical lock files ${HALFPIPE_OUTPUTBASE}/lock/halfpipe-${HALFPIPE_DOWNLINKID} .  Have to check when this file is being removed.
  • RetDef files $HALFPIPE_OUTPUTBASE/$HALFPIPE_DOWNLINKID/RetDef-$HALFPIPE_DOWNLINKID-$HALFPIPE_CHUNKID.xml are missing.  u42 isn't on s3df.  We copied these files from rhel6-64:/nfs/farm/g/glast/u42/ISOC-flight/Downlinks/<dl>/ .
• mergeIndices.sh
  • tokendir is read dinamically: tokendir=`cat ${taskBase}/config/stagedir`/chunktokens . For testing we changed stagedir from /nfs/farm/g/glast/u28/stage to /sdf/group/fermi/a/isoc/flightOps/volumes/vol5/offline/halfPipe/v7r0p0_devel/u28/stage .
  • at the end a mergeEvt stream for each run is created.  This won't work in the container.  This task is being taken by launchMerge , an embedded scriptlet in the xml.
• launchMerge: embedded scriptlet to launch mergeEvt
• mergeEvt.sh
  • should receive ${HALFPIPE_RUNSTART} from launchMerge.  Doesn't work right now.  Workaround: read the decimal runid from PIPELINE_STREAM and convert to hex.  Michael thinks this should be safe.
  • not fault of mergeEvt, I guess: mergeEvt runs always on sdfmilan11[56].  There is no /lscratch but /scratch , and /sdf/group/fermi/a/isoc/flightOps/volumes/vol5/offline/halfPipe/v7r0p0_devel/outputBase is mapped to uid 7234 , while glastraw is 9503 .  How to report this?
  • Michael did something else yesterday to make mergeEvt finish.  But probably the above should be understood and fixed first.