...
Run Number | Reason | Date | Time | Dls | L1Proc | GR version | rePipe | L1Proc |
---|---|---|---|---|---|---|---|---|
565721985 | Botched attempt at a repipe, repiped and marked as Complete | 2018-12-05 | L1Proc-5.7 | GlastRelease-20-10-04-gr02 | ||||
565942990 | stuck "in progress", repiped and marked as Complete | 2018-12-08 | L1Proc-5.7 | GlastRelease-20-10-04-gr02 | ||||
566022500 | HP "in progress", repiped and marked as Complete | 2018-12-09 | L1Proc-5.7 | GlastRelease-20-10-04-gr02 | ||||
566235626 | HP "in progress", repiped and marked as Complete | 2018-12-11 | L1Proc-5.7 | GlastRelease-20-10-04-gr02 | ||||
567301141 | stuck "in progress", repiped and marked as Complete | 2018-12-23 | L1Proc-5.7 | GlastRelease-20-10-04-gr02 | ||||
567598970 | Missing Data - Delivery 181227007 couldn't be dispatched - Marked as Incomplete and finalized. | 2018-12-27 | L1Proc-5.7 | GlastRelease-20-10-04-gr02 | ||||
567581889 | Missing Data - Delivery 181227007 couldn't be dispatched - Marked as Incomplete and finalized. | 2018-12-27 | L1Proc-5.7 | GlastRelease-20-10-04-gr02 | ||||
569069307 | in progress and errors about missing datagrams (MK: update: data complete but no recon). MEM: rolled back checkRun and finalized. | 2019-01-13 | 010 011 | L1Proc-5.7 | GlastRelease-20-10-04-gr02 | |||
569213577 | in progress and errors about missing datagrams (MK: update: data complete but no recon). MEM: rolled back checkRun and finalized. | 2019-01-15 | 006 007 | L1Proc-5.7 | GlastRelease-20-10-04-gr02 | |||
572617673 | HP "in progress", repiped and marked as Complete | 2019-02-23 | L1Proc-5.7 | GlastRelease-20-10-04-gr02 | ||||
572853518 | stuck "in progress". Leftover chunkToken. Fixed. | 2019-02-26 | L1Proc-5.7 | GlastRelease-20-10-04-gr02 | ||||
574781844 | "in progress", incomplete data, missing datagrams | 2019-03-20 | L1Proc-5.7 | GlastRelease-20-10-04-gr02 | ||||
575044322 | "in progress", incomplete data, missing datagrams (MK: update: stat is InProgress, no recon file in db). MEM: apparently the halfPipe was rolled back after run was already reprocessed. Re-set as Complete and finalized. | 2019-03-23 | 013 014 | L1Proc-5.7 | GlastRelease-20-10-04-gr02 | |||
575762229 | "in progress", incomplete data, missing datagrams | 2019-03-31 | L1Proc-5.7 | GlastRelease-20-10-04-gr02 | ||||
576305031 | Run was terminated by the "LAT configuration corruption" event. Marked as Incomplete and finalized. | 2019-04-07 | L1Proc-5.7 | GlastRelease-20-10-04-gr02 | ||||
576658880 | "in progress", incomplete data | 2019-04-11 | L1Proc-5.7 | GlastRelease-20-10-04-gr02 | ||||
578530168 | corrupted data in monitoring | 2019-05-02 | L1Proc-5.7 | GlastRelease-20-10-04-gr02 | ||||
580907445 | in progress, mergeIndices reports 5 missing datagrams | 2019-05-30 | L1Proc-5.7 | GlastRelease-20-10-04-gr02 | ||||
582073028 | in progress, mergeIndices reports missing datagrams | 2019-06-13 | L1Proc-5.7 | GlastRelease-20-10-04-gr02 | ||||
584730839 | in progress, complete data, missing datagrams | 2019-07-13 | L1Proc-5.7 | GlastRelease-20-10-04-gr02 | ||||
585187986 | in progress, complete data, missing datagrams, problems with rolling back halfPipe.190719003 | 2019-07-19 | L1Proc-5.7 | GlastRelease-20-10-04-gr02 | ? | |||
585193691 | in progress, complete data, missing datagrams, problems with rolling back halfPipe.190719003 | 2019-07-19 | L1Proc-5.7 | GlastRelease-20-10-04-gr02 | ||||
585535991 | in progress, complete data, missing datagrams | 2019-07-23 | L1Proc-5.7 | GlastRelease-20-10-04-gr02 | ||||
585690096 | in progress, missing datagrams | 2019-07-24 | L1Proc-5.7 | GlastRelease-20-10-04-gr02 | ||||
586076888 | see thread. However it seems the recon file is complete. Reprocessing complete. Everything seems to have worked including the cleanupCompleteRun process. | 2019-07-29 | L1Proc-5.7 | GlastRelease-20-10-04-gr02 | ||||
586893790 | bunch of tasks in "Waiting" state preventing a rollback. Once that is cleared a rollback should hopefully fix everything | 2019-08-07 | L1Proc-5.7 | GlastRelease-20-10-04-gr02 | N/A | |||
586905200 | see thread . Seems the run is complete and can be just finalized. | 2019-08-07 | L1Proc-5.7 | GlastRelease-20-10-04-gr02 | ||||
587042253 | Several failed tasks with errors of the form: "IOError: Can't stage in <filename>". Looks like it was an xrootd hiccup. Will try rolling back but since it's so old, it will probably require a repipe. Rollback worked now complete | 2019-08-09 12:46 | L1Proc-5.7 | GlastRelease-20-10-04-gr02 | N/A | |||
587104373 | This one just looks like cleanupCompleteRun didn't run properly as everything else in both parts of the runs was listed as success. There is a chunk token directory but no token. However, the verify log complains about closing reasons for both EUP0 and EPU1. I'm think that means we need to repipe everything. | 2019-08-10 6:54 | L1Proc-5.7 | GlastRelease-20-10-04-gr02 | ||||
588742217 | Run is incomplete according to the halfPipe. | 2019-08-29 | L1Proc-5.7 | GlastRelease-20-10-04-gr02 | ||||
590065315 | Run still Incomplete in L1. Looks like cleanupCompleteRun didn't get triggered. | 2019-09-13 | L1Proc-5.7 | GlastRelease-20-10-04-gr02 | ||||
591931054 | failure in checkRun 191005003.591931054. 2 missing datagram. | 2019-10-05 | L1Proc-5.7 | GlastRelease-20-10-04-gr02 | ||||
595240349 | 9 missing datagrams | 2019-11-12 | L1Proc-5.7 | GlastRelease-20-10-04-gr02 | ||||
595405654 | missing datagrams | 2019-11-14 | L1Proc-5.7 | GlastRelease-20-10-04-gr02 | ||||
595576882 | missing datagrams. In progress | 2019-11-16 | L1Proc-5.7 | GlastRelease-20-10-04-gr02 | ||||
595963106 | missing datagrams - The repiped run report more missing data than the original run. I'm at a loss on how to fix this one. | 2019-11-20 | 17:18:24 - 18:32:18 | 191120013 191121001 | L1Proc-5.7 | GlastRelease-20-10-04-gr02 | ||
596299562 | HalfPipe reports "inProgress", gapfinder doesn't report any data gaps for 2019-11-24 21:30-24:00 | 2019-11-24 | 14:46:00 - 16:01:21 | 191124009 | L1Proc-5.7 | GlastRelease-20-10-04-gr02 | ||
596916997 | missing datagrams | 2019-12-01 | L1Proc-5.7 | GlastRelease-20-10-04-gr02 | ||||
597315084 | HalfPipe reports "inProgress", gapfinder doesn't report any data gaps for 2019-12-06 05:00-14:00 | 2019-12-06 | 08:51:23 - 10:06:10 | 191206009 010 | L1Proc-5.7 | GlastRelease-20-10-04-gr02 | ||
598223809 | HalfPipe reports "inProgress", missing datagrams, gapfinder doesn't report any data gaps for 2019-12-16 | 2019-12-16 | 21:16:48 - 22:51:39 | 191217001 002 | L1Proc-5.7 | GlastRelease-20-10-04-gr02 | ||
598581385 | HalfPipe reports "inProgress", missing datagrams, gapfinder doesn't report any data gaps for 2019-12-21 | 2019-12-21 | 00:36:24 - 01:53:08 | 191221003 | L1Proc-5.7 | GlastRelease-20-10-04-gr02 | ||
598645966 | HalfPipe reports "inProgress", missing datagrams, gapfinder doesn't report any data gaps for 2019-12-21 | 2019-12-21 | 18:32:45 - 20:07:36 | 191221009 | L1Proc-5.7 | GlastRelease-20-10-04-gr02 | ||
598758794 | HalfPipe reports "inProgress" | 2019-12-23 | 05:39:57 - 08:44:23 | 191223003 004 | L1Proc-5.7 | GlastRelease-20-10-04-gr02 | ||
599272563 | HalfPipe reports "inProgress". Datagram errors | 2019-12-29 | 2:22:25 - 4:58:04 | 191229002 003 | L1Proc-5.7 | GlastRelease-20-10-04-gr02 | ||
599746995 | HalfPipe reports "inProgress", missing datagrams, gapfinder doesn't report any data gaps for 2020-01-03 | 2020-01-03 | 12:23:14 - 13:58:05 | 200103009 010 011 | L1Proc-5.7 | GlastRelease-20-10-04-gr02 | ||
599809968 | HalfPipe reports "inProgress", missing datagrams, gapfinder doesn't report any data gaps for 2020-01-04 | 2020-01-04 | 05:52:47 - 07:14:52 | 200103009 010 011 | L1Proc-5.7 | GlastRelease-20-10-04-gr02 | ||
600328891 | HalfPipe reports "inProgress". Datagram errors | 2020-01-10 | 10:43:13 | 200110005 | L1Proc-5.7 | GlastRelease-20-10-04-gr02 | ||
601777922 | HalfPipe reports "inProgress", but mergeEvt doesn't report any datagram errors! Gapfinder doesn't report any data gaps for 2020-01-27. | 2020-01-27 | 02:07:06 - 03:41:57 | 200127002 003 004 | L1Proc-5.7 | GlastRelease-20-10-04-gr02 | ||
601783627 | HalfPipe reports "inProgress". Missing datagrams. Gapfinder doesn't report any data gaps for 2020-01-27. | 2020-01-27 | 00:32:01 - 02:06:55 | 200127004 005 006 | L1Proc-5.7 | GlastRelease-20-10-04-gr02 | ||
601890911 | HalfPipe reports "inProgress". Missing datagrams. Gapfinder doesn't report any data gaps for the morning of 2020-01-28. | 2020-01-28 | 07:55:10 - 09:04:44 | 200128006 008 | L1Proc-5.7 | GlastRelease-20-10-04-gr02 | ||
602153762 | HalfPipe reports "inProgress". Missing datagrams. Gapfinder doesn't report any data gaps for 2020-01-31. No decompression errors in makeEvt. | 2020-01-31 | 08:56:01 - 10:07:38 | 200131007 | L1Proc-5.7 | GlastRelease-20-10-04-gr02 | ||
603637679 | HalfPipe reports "inProgress". Missing datagrams. Gapfinder doesn't report any data gaps for 2020-02-17. No decompression errors in makeEvt. | 2020-02-17 | 13:07:57 - 14:42:49 | 200217009 010 | L1Proc-5.7 | GlastRelease-20-10-04-gr02 | ||
603974262 | Run is complete. mergeEvt.200221009.603974262 reports 6 missing datagrams. gapfinder doesn't report any data gaps for the time interval. | 2020-02-21 | 10:37:41 - 12:12:31 | 200221009 | L1Proc-5.7 | GlastRelease-20-10-04-gr02 | ||
604573262 | HalfPipe reports "inProgress". Missing datagrams. Gapfinder doesn't report any data gaps for 2020-02-28. | 2020-02-28 | 09:01:01 - 10:35:52 | 20200228008 - 009 | L1Proc-5.7 | GlastRelease-20-10-04-gr02 |
|
...