...
Run Number | Reason | Date | Time | Dls | L1Proc | GR version | rePipe | L1Proc |
---|---|---|---|---|---|---|---|---|
743660123 | MergeEvt found a gap of 6 datagrams in epu1. Nevertheless the halfPipe labeled the run as complete. Fortunately L1Proc.verify choked on it. Gapfinder doesn't report any gaps. | 2024-07-26 | 240716004 | L1Proc-5.9 | GlastRelease-20-10-04-gr07 | |||
742704925 | Puzzling. MergeEvt seems to have found all datagrams, but nevertheless labels both epu's as incomplete. L1Proc.verify claims that one datagram was dropped in epu0. | 2024-07-15 | 24071500[34] | L1Proc-5.9 | GlastRelease-20-10-04-gr07 | |||
741148235 | MergeEvt reports 6 missing datagrams in epu1. Gapfinder doesn't report a gap for the duration of this run. | 2024-06-27 | 24062700[34] | L1Proc-5.9 | GlastRelease-20-10-04-gr07 | |||
738549526 | L1Proc.verify claims to have found gaps of 5 datagrams each in both epus. mergeEvt reports the gap for epu0 but claims epu1 is complete. The HalfPipe labeled the run complete. Gapfinder doesn't report a gap for the duration of this run. | 2024-05-31 | 24053100[45] | L1Proc-5.9 | GlastRelease-20-10-04-gr07 | |||
738549526 | mergeEvt reports 7 missing datagrams in epu1. Gapfinder doesn't report a gap for the duration of this run. | 2024-05-28 | 24052800[123] | L1Proc-5.9 | GlastRelease-20-10-04-gr07 | |||
736930436 | mergeEvt reports 6 missing datagrams in epu0. Gapfinder doesn't report a gap for the duration of this run. | 2024-05-09 | 24050900[89] | L1Proc-5.9 | GlastRelease-20-10-04-gr07 | |||
730458995 | 240224011.mergeEvt ran twice, creating an incomplete evt file on the second try. | 2024-02-24 | 24022401[01] | L1Proc-5.9 | GlastRelease-20-10-04-gr07 | |||
729548905 | Highly fragmented run. 91, 15, and 61 chunks. This requires task RePipe. | 2024-02-14 | 24021400[345] | L1Proc-5.9 | GlastRelease-20-10-04-gr07 | |||
727336262 | mergeEvt reports 6 missing datagrams in epu1. Gapfinder doesn't report a gap for the duration of this run. | 2024-01-19 | 240119003 | L1Proc-5.9 | GlastRelease-20-10-04-gr07 | |||
727324474 | mergeEvt reports missing datagrams at the end of epu1. Gapfinder doesn't report a gap for the duration of this run. | 2024-01-19 | 240119002 | L1Proc-5.9 | GlastRelease-20-10-04-gr07 | |||
726579387 | mergeEvt reports 5 missing datagrams in epu1. Gapfinder doesn't report a gap for the duration of this run. | 2024-01-10 | 24011000[78] | L1Proc-5.9 | GlastRelease-20-10-04-gr07 | |||
726452942 | mergeEvt reports missing datagrams at the end of epu1. Gapfinder doesn't report a gap for the duration of this run. | 2024-01-09 | 00:29:01 - 01:41:00 | 240109004 | L1Proc-5.9 | GlastRelease-20-10-04-gr07 | ||
723956756 | mergeEvt reports 6 missing datagrams in epu0. Gapfinder doesn't report a gap for the duration of this run. | 2023-12-11 | 03:05:55 - 04:39:36 | 23121100[23] | L1Proc-5.9 | GlastRelease-20-10-04-gr07 | ||
723939304 | mergeEvt reports 5 missing datagrams in epu1. Gapfinder doesn't report a gap for the duration of this run. | 2023-12-10 | 22:15:02 - 23:32:30 | 231211001 | L1Proc-5.9 | GlastRelease-20-10-04-gr07 | ||
723865764 | mergeEvt reports 6 missing datagrams in epu1. Gapfinder doesn't report any gaps. | 2023-12-10 | 23121000[34] | L1Proc-5.9 | GlastRelease-20-10-04-gr07 | |||
723665322 | mergeEvt reports 6 missing datagrams in epu1. Gapfinder found a gap for this run and sent just one retransmit request. | 2023-12-07 | 18:08:41 - 19:19:22 | 23120101[02] | L1Proc-5.9 | GlastRelease-20-10-04-gr07 | ||
723103279 | mergeEvt reports about 100 missing datagrams at the end of epu1. Gapfinder doesn't report any gaps. | 2023-12-01 | 23120100[78] | L1Proc-5.9 | GlastRelease-20-10-04-gr07 | |||
717719650 | Dls 230929007 and 009 resulted from passes the FOT called "ratty". All run pieces in these dls were highly fragmented, and L1 didn't process them. Redumps were triggered, but only run 717708266 was completely filled in dl 230930008, 717713958 partially. Of the latter it's estimated that about half of the data are missing (1.2M events), while 717702333 (2.5M events) and 717719650 (2.7M events) should be almost filled. UPDATE: rePipe didn't result in more merged events. The runs were finalized as is, i.e. with the gaps. | 2023-09-29 | 23093000[123], (?230929009?) | L1Proc-5.9 | GlastRelease-20-10-04-gr07 | - | ||
717713958 | 2023-09-29 | 230930008, (?230929009?) | L1Proc-5.9 | GlastRelease-20-10-04-gr07 | - | |||
717702333 | 2023-09-29 | 230929008, 23093000[12], (?230929007?) | L1Proc-5.9 | GlastRelease-20-10-04-gr07 | - | |||
710506750 | mergeEvt reports 6 missing datagrams in epu0. Gapfinder doesn't report any gaps. | 2023-07-08 | 23070800[89] | L1Proc-5.9 | GlastRelease-20-10-04-gr07 | |||
709339013 | somehow the xrootd scratch area vanished before cleanupCompleteRun. The run needs to be repiped from the HalfPipe. | 2023-06-24 | 22:41:52 - 00:16:33 (+1) | 23062500[12] | L1Proc-5.9 | GlastRelease-20-10-04-gr07 | ||
708257402 | mergeEvt reports 8 missing datagrams in epu1. Gapfinder doesn't report any gaps. | 2023-06-12 | 23031200[56] | L1Proc-5.9 | GlastRelease-20-10-04-gr07 | |||
700230335 | mergeEvt reports 5 missing datagrams in epu0. Gapfinder doesn't report any gaps. | 2023-02-11 | 230311009 | L1Proc-5.9 | GlastRelease-20-10-04-gr07 | |||
697691485 | mergeEvt reports 7 missing datagrams in epu1. | 2023-02-10 | 23021000[45] | L1Proc-5.9 | GlastRelease-20-10-04-gr07 | |||
694642846 | verify found a datagram gap. Actually already mergeEvt found this gap of 12 datagrams in epu1, but somehow the run was labeled as complete. A HalfPipe repipe leaves one missing datagram, decompression error. Finalized with one datagram gap. | 2023-01-05 | 230105010, 230106001 | L1Proc-5.9 | GlastRelease-20-10-04-gr07 | - | ||
693360593 | 003.mergeIndices failed and was processed last. When mergeEvt ran it didn't merge in the datagrams of 004. | 2022-12-22 | 22122200[234] | L1Proc-5.9 | GlastRelease-20-10-04-gr07 | |||
687193623 | mergeReconChunks failed with a bus error but exit code 0. The merged recon file was not copied to xrootd but cleanupChunks removed all temporary files. | 2022-10-11 | 22101101[123] | L1Proc-5.9 | GlastRelease-20-10-04-gr07 | |||
687119520 | mergeEvt reports 6 missing datagrams in epu0, gapfinder doesn't report any gaps. | 2022-10-10 | 221010013 | L1Proc-5.9 | GlastRelease-20-10-04-gr07 | |||
686035262 | dl's 220928003 and 004 came in together. It was not possible to disentangle both. 220928003.mergeIndices fails. | 2022-09-28 | 22092800[34] | L1Proc-5.9 | GlastRelease-20-10-04-gr07 | |||
68602???? | 2022-09-28 | 220928003 | L1Proc-5.9 | GlastRelease-20-10-04-gr07 | ||||
686022685 | 2022-09-28 | 22092800[23] | L1Proc-5.9 | GlastRelease-20-10-04-gr07 | ||||
685255472 | mergeEvt reports 7 missing datagrams in epu0, gapfinder doesn't report any gaps. | 2022-09-19 | 22091900[45] | L1Proc-5.9 | GlastRelease-20-10-04-gr07 | |||
682569294 | Aug 19 (PDT) there were issues with a faulty nfs disk. This run was merged badly. | 2022-08-19 | 02:34:53 - 03:45:03 | 22081900[45] | L1Proc-5.9 | GlastRelease-20-10-04-gr07 | ||
682556983 | Aug 19 (PDT) there were issues with a faulty nfs disk. This run was merged badly. | 2022-08-18 | 23:09:42 - 00:29:45 (+1) | 22081900[34] | L1Proc-5.9 | GlastRelease-20-10-04-gr07 | ||
680740571 | mergeEvt reports 5 missing datagrams in epu1, gapfinder doesn't report any gaps. | 2022-07-28 | 22:36:10 - 23:58:01 | 22072900[12] | L1Proc-5.9 | GlastRelease-20-10-04-gr07 | ||
676577929 | mergeEvt reports 5 missing datagrams in epu0, gapfinder doesn't report any gaps. | 2022-06-10 | 22061001[01] | L1Proc-5.9 | GlastRelease-20-10-04-gr07 | |||
675367448 | mergeEvt reports 8 missing datagram in epu0, additionally verify finds 7 missing datagrams in epu1. However gapfinder doesn't report any gaps. | 2022-05-27 | 18:04:07 - 19:28:29 | 220527012, 220528001 | L1Proc-5.9 | GlastRelease-20-10-04-gr07 | ||
674884484 | mergeEvt reports 1 missing datagram in epu1, gapfinder doesn't report any gaps. MakeEvt reports a decompression error, finalizing with one datagram gap. | 2022-05-22 | 2205220[567] | L1Proc-5.9 | GlastRelease-20-10-04-gr07 | - | ||
674878528 | mergeEvt reports several gaps with missing datagrams in epu1, gapfinder doesn't report any gaps. MakeEvt reports decompression errors, finalizing with 14 scattered datagram gaps. | 2022-05-22 | 22052200[45] | L1Proc-5.9 | GlastRelease-20-10-04-gr07 | - | ||
674872543 | mergeEvt reports several gaps with missing datagrams in epu1, gapfinder doesn't report any gaps. MakeEvt reports decompression errors, finalizing with 16 missing datagrams scattered over 14 gaps. | 2022-05-22 | 22052200[34] | L1Proc-5.9 | GlastRelease-20-10-04-gr07 | - | ||
674052578 | mergeEvt reports 5 missing datagrams in epu1, gapfinder doesn't report any gaps. | 2022-05-12 | 220512010 | L1Proc-5.9 | GlastRelease-20-10-04-gr07 | |||
671827689 | verify reported missing data. Indeed also mergeEvt reports 5 missing datagrams in epu0, but somehow the run was labeled as complete. | 2022-04-16 | 220416014 | L1Proc-5.9 | GlastRelease-20-10-04-gr07 | |||
671754063 | missing datagrams at the end of epu1. gapfinder doesn't report any gaps. | 2022-04-16 | 22041600[23] | L1Proc-5.9 | GlastRelease-20-10-04-gr07 | |||
670363411 | Dls 220331005 and 006 came in within the same second, and got mangled. 006 passed, but 005 fails at HalfPipe.mergeIndices. Needs to be repiped completely. | 2022-03-31 | 22033100[25] | L1Proc-5.9 | GlastRelease-20-10-04-gr07 | |||
670357708 | Dls 220331005 and 006 came in within the same second, and got mangled. 006 passed, but 005 fails at HalfPipe.mergeIndices. Needs to be repiped completely. | 2022-03-31 | 22033100[345] | L1Proc-5.9 | GlastRelease-20-10-04-gr07 | |||
661643650 | Highly fragmented run (138 chunks). Needs a "real" repipe. | 2021-12-20 | 21:54:09 - 23:28:58 | 21122000[124] | L1Proc-5.9 | GlastRelease-20-10-04-gr07 | ||
656744433 | Fall-out of the 211024 hypervisor failure. EPU0 is complete but EPU1 has a gap of ~5000 datagrams. Roll back of HalfPipe didn't work! Repipe did. But there was a gap in the magic7 file → second repipe | 2021-10-24 | 21102400[56] | L1Proc-5.9 | GlastRelease-20-10-04-gr07 | |||
649557766 | Highly fragmented run (>400 chunks). Needs a "real" repipe. | 2021-08-02 | 21080200[234] | L1Proc-5.9 | GlastRelease-20-10-04-gr07 | |||
648918184 | mergeEvt reports a gap of 5 datagrams in epu0. Gapfinder doesn't report any gaps. | 2021-07-25 | 15:03:03 - 16:13:05 | 21072501[01] | L1Proc-5.9 | GlastRelease-20-10-04-gr07 | ||
648862027 | mergeEvt reports a gap of 5 datagrams in epu1. Gapfinder doesn't report any gaps. | 2021-07-24 | 23:27:06 - 00:59:49[+1] | 21072500[12] | L1Proc-5.9 | GlastRelease-20-10-04-gr07 | ||
645896054 | One datagram in the inaccessible part of the SSR. The run will stay incomplete. | 2021-06-20 | 21062001[12] | L1Proc-5.9 | GlastRelease-20-10-04-gr07 | - | ||
645331200 | mergeEvt reports a gap of 9 datagrams in epu1. Gapfinder doesn't report any gaps. | 2021-06-14 | 21061400[34] | L1Proc-5.9 | GlastRelease-20-10-04-gr07 | |||
644150495 | mergeEvt reports a gap of 9 datagrams in epu0. Gapfinder doesn't report any gaps. | 2021-05-31 | 21053100[78] | L1Proc-5.9 | GlastRelease-20-10-04-gr07 |
...