...
- Data arrives from GSFC
- L0 ingest triggered by FastCopy post-processing script (into database)
Fast copy: https://confluence.slac.stanford.edu/display/ISOC/FASTCopy+processing+chain
processsci.py (?? I didn't really catch this)
- Launches gap finder to look for missing pieces
- Divides delivery into chunks and writes xml file
- Up to 20 chunks/delivery. Usually ~12 or so
- Halfpipe keeps track of what has been delivered
- Once tidy & ordered, hands off to L1
- L1 registers in data catalog & sends FITS to GSFC
...
- Current shifting scheme:
- Warren & Michael each take responsibility for 1/2 day.
- Maria Elena covers for when Warren on vacation.
- No coverage for Michael.
- Large disorganized document
https://confluence.slac.stanford.edu/display/ds/Things+to+know+while+on-call+for+Data+Processing- Needs to be split into child pages and updated
- As new person gets trained, would be good to start working on reorganization
- Some work done via command line to interact with LSF.
- Most (95%) done via the web interface.
- Luca L. was looking at adding pipeline monitoring for young Torino folks.
Halfpipe (Steve Tether, maybe Gregg?):
Halfpipe has a specific Operations Task - gap reporting? (from Rob)
Warren: No, occurs before the Halfpipe…during FastCopy post-processing
Fast copy: https://confluence.slac.stanford.edu/display/ISOC/FASTCopy+processing+chain
processsci.py (?? I didn't really catch this)
...
- LSF.
- Most (95%) done via the web interface.
- Luca L. was looking at adding pipeline monitoring for young Torino folks.
Halfpipe (Steve Tether, maybe Gregg?):
Halfpipe has a specific Operations Task - gap reporting? (from Rob)
Warren: No, occurs before the Halfpipe…during FastCopy post-processing
...
Halfpipe reads chunked files:
...