...
If they are in a Failed state, you can just do a standard rollback.
If findChunks is stuck, then bkill the process and wait for the reaper to get it. Most of them should then auto rollback without intervention.
History
For instance, this error in ft2Runs stream 240415006.734877891:
Code Block | ||
---|---|---|
| ||
> terminate called after throwing an instance of 'std::runtime_error'
> what(): FATAL: the provided Magic 7 file does not cover the requested time interval. To cover the requested interval we would need to extrapolate position and attitude (forward) more than what permitted by the current configuration (see the parameter 'extrapolationLimit'). |
Michael fixed this by:
1) doRun.ft2Runs (case of 240415006.734877891) reads from the runs area:
stageIn for: /nfs/farm/g/glast/u41/L1/runs/734/r0734877891/r0734877891_v000_magic7L1.txt
The magic7 file in the staging area, which is also being copied into the
runs area, is complete. makeM7L1 reads from the staging area.
Thus, I just rolled back makeM7L1, which found all packets and created a
valid magic7L1 file in the runs area, to be read by ft2Runs.
2) doRun.doChunk.fakeFT2 (case of 240414007.734792595.6674757) stages from
the staging area:
stageIn for: /nfs/farm/g/glast/u28/stage/240414007/magic7_240414007.txt
This file was incomplete! I replaced it by the 240414008 magic7 file
PGWave reports a duplicate stream error, e.g.,
Code Block | ||
---|---|---|
| ||
Task drpMonitoring Process launchDrpMonitoring Stream 745027200.0
org.srs.pipeline.server.sql.DatabaseUtilities$DuplicateStreamException: A stream ALREADY exists with specified task, parent, and id |
According to Jim, if the downstream drpMonitoring task has already managed to be submitted, the failed PGWave.launchDrpMonitoring stream can be left as-is. If you want to clean it up, temporarily disable the code to launch the drpMoninitoring stream and then roll it back.