Run Number | Reason | Date | Dls | L1Proc | GR version | rePipe | L1Proc |
---|---|---|---|---|---|---|---|
565721985 | Botched attempt at a repipe, repiped and marked as Complete | 2018-12-05 | L1Proc-5.7 | GlastRelease-20-10-04-gr02 | |||
565942990 | stuck "in progress", repiped and marked as Complete | 2018-12-08 | L1Proc-5.7 | GlastRelease-20-10-04-gr02 | |||
566022500 | HP "in progress", repiped and marked as Complete | 2018-12-09 | L1Proc-5.7 | GlastRelease-20-10-04-gr02 | |||
566235626 | HP "in progress", repiped and marked as Complete | 2018-12-11 | L1Proc-5.7 | GlastRelease-20-10-04-gr02 | |||
567301141 | stuck "in progress", repiped and marked as Complete | 2018-12-23 | L1Proc-5.7 | GlastRelease-20-10-04-gr02 | |||
567598970 | Missing Data - Delivery 181227007 couldn't be dispatched - Marked as Incomplete and finalized. | 2018-12-27 | L1Proc-5.7 | GlastRelease-20-10-04-gr02 | |||
567581889 | Missing Data - Delivery 181227007 couldn't be dispatched - Marked as Incomplete and finalized. | 2018-12-27 | L1Proc-5.7 | GlastRelease-20-10-04-gr02 | |||
569069307 | in progress and errors about missing datagrams (update: data complete but no recon) | 2019-01-13 | 010 011 | L1Proc-5.7 | GlastRelease-20-10-04-gr02 | ?? | |
569213577 | in progress and errors about missing datagrams (update: data complete but no recon) | 2019-01-15 | 006 007 | L1Proc-5.7 | GlastRelease-20-10-04-gr02 | ?? | |
572617673 | HP "in progress", repiped and marked as Complete | 2019-02-23 | L1Proc-5.7 | GlastRelease-20-10-04-gr02 | |||
572853518 | stuck "in progress". Leftover chunkToken. Fixed. | 2019-02-26 | L1Proc-5.7 | GlastRelease-20-10-04-gr02 | |||
574781844 | "in progress", incomplete data, missing datagrams | 2019-03-20 | L1Proc-5.7 | GlastRelease-20-10-04-gr02 | |||
575044322 | "in progress", incomplete data, missing datagrams (update: stat is InProgress, no recon file in db) | 2019-03-23 | 013 014 | L1Proc-5.7 | GlastRelease-20-10-04-gr02 | ?? | |
575762229 | "in progress", incomplete data, missing datagrams | 2019-03-31 | L1Proc-5.7 | GlastRelease-20-10-04-gr02 | |||
576305031 | Run was terminated by the "LAT configuration corruption" event. Marked as Incomplete and finalized. | 2019-04-07 | L1Proc-5.7 | GlastRelease-20-10-04-gr02 | |||
576658880 | "in progress", incomplete data | 2019-04-11 | L1Proc-5.7 | GlastRelease-20-10-04-gr02 | |||
578530168 | corrupted data in monitoring | 2019-05-02 | L1Proc-5.7 | GlastRelease-20-10-04-gr02 | |||
580907445 | in progress, mergeIndices reports 5 missing datagrams | 2019-05-30 | L1Proc-5.7 | GlastRelease-20-10-04-gr02 | |||
582073028 | in progress, mergeIndices reports missing datagrams | 2019-06-13 | L1Proc-5.7 | GlastRelease-20-10-04-gr02 | |||
584730839 | in progress, complete data, missing datagrams | 2019-07-13 | L1Proc-5.7 | GlastRelease-20-10-04-gr02 | |||
585187986 | in progress, complete data, missing datagrams, problems with rolling back halfPipe.190719003 | 2019-07-19 | L1Proc-5.7 | GlastRelease-20-10-04-gr02 | |||
585193691 | in progress, complete data, missing datagrams, problems with rolling back halfPipe.190719003 | 2019-07-19 | L1Proc-5.7 | GlastRelease-20-10-04-gr02 | |||
585535991 | in progress, complete data, missing datagrams | 2019-07-23 | L1Proc-5.7 | GlastRelease-20-10-04-gr02 | |||
585690096 | in progress, missing datagrams | 2019-07-24 | L1Proc-5.7 | GlastRelease-20-10-04-gr02 | |||
586076888 | see thread. However it seems the recon file is complete | 2019-07-29 | L1Proc-5.7 | GlastRelease-20-10-04-gr02 | Tom | ||
586893790 | bunch of tasks in "Waiting" state preventing a rollback. Once that is cleared a rollback should hopefully fix everything | 2019-08-07 | L1Proc-5.7 | GlastRelease-20-10-04-gr02 | |||
586905200 | see thread. Seems the run is complete and can be just finalized. After checking, I've finalized this run as it looks fine -TS | 2019-08-07 | L1Proc-5.7 | GlastRelease-20-10-04-gr02 | |||
588742217 | Run is incomplete according to the halfPipe | 2019-08-29 | L1Proc-5.7 | GlastRelease-20-10-04-gr02 | I think this is okay as I don't see any errors in the verify log. |
/afs/slac/g/glast/ground/bin/pipeline createStream --define "downlinkID=990101001,runID=237063408" RePipe
/nfs/farm/g/glast/u28/RePipe/990101001 will contain the RePiped output (should this have a --stream <run number> flag in the command for clarity in the monitoring pages?)
More details: Instructions for rolling back a halfPipe stream and susbsequent L1 stream and Reprocessing runs from a RePiped job
Run Number | Reason | Date | L1Proc | GR version | rePipe | L1(Re)Proc | Export |
---|---|---|---|---|---|---|---|
369152685 | ACQ Complete, L1 not finalized. Fixed in P300x. | 2012-09-12 | L1Proc-4.0 | GR-17-35-24-gr35 | 4.0 | ||
478706331 | ACQ Complete, L1 not finalized. Fixed in P300x. | 2016-03-03 | L1Proc-5.4 | GR-20-10-02 | 5.4 | ||
478756313 | ACQ Complete, L1 not finalized. Fixed in P300x. | 2016-03-04 | L1Proc-5.4 | GR-20-10-02 | 5.4 | ||
504919670 | Leap Second Run. Set to Incomplete and finalized. | 2016-12-31 | L1Proc-5.6 | GR-20-10-04-gr01 | 5.6 | ||
508046861 | ACQ InProgress. Repiped and reprocessed. | 2017-02-09 | L1Proc-5.6 | GR-20-10-04-gr01 | 5.7 | ||
512115535 | ACQ Complete, L1 Complete/Failed. Fixed in P300x. | 2017-03-25 | L1Proc-5.6 | GR-20-10-04-gr01 | 5.6 | ||
516617903 | ACQ Complete, L1 Complete/Failed. Fixed in P300x. | 2017-05-16 | L1Proc-5.6 | GR-20-10-04-gr01 | 5.6 | ||
531419768 | ACQ Complete, L1 Complete, recon on xrootd (root://fermi-gpfs05//glast/Data/Flight/Level1/LPA/prod/5.6/recon/r0531419768_v000_recon.root) but not in datacat | 2017-11-03 | L1Proc-5.6 | GR-20-10-04-gr01 | |||
534500370 | ACQ Complete, L1 Complete/Failed. Fixed in P300x. | 2017-12-09 | L1Proc-5.6 | GR-20-10-04-gr01 | 5.6 | ||
549415721 | ACQ Complete, L1 Complete, recon on xrootd (root://fermi-gpfs05//glast/Data/Flight/Level1/LPA/prod/5.6/recon/r0549415721_v000_recon.root) but not in datacat | 2018-05-31 | L1Proc-5.6 | GR-20-10-04-gr01 | |||
553318321 | ACQ Complete, L1 Failed. Repiped and reprocessed. | 2018-07-16 | L1Proc-5.6 | GR-20-10-04-gr01 | 5.7 | ||
554179852 | ACQ Complete, L1 Failed. Repiped and reprocessed. | 2018-07-25 | L1Proc-5.6 | GR-20-10-04-gr01 | 5.7 | ||
554396875 | ACQ Complete, cleanupCompleteRun.getTimes terminated by reaper | 2018-07-27 | L1Proc-5.6 | GR-20-10-04-gr01 | |||
555737292 | ACQ Complete, L1 InProgress. Finalized. | 2018-08-12 | L1Proc-5.6 | GR-20-10-04-gr01 | 5.6 | ||
557026900 | ACQ Complete, L1 InProgress. Finalized. | 2018-08-27 | L1Proc-5.6 | GR-20-10-04-gr01 | 5.6 | ||
559965007 | Truncated Run. Set to Incomplete and finalized. | 2018-09-30 | L1Proc-5.6 | GR-20-10-04-gr01 | 5.6 | ||
562253064 | Missing Data. Set to Incomplete and finalized. | 2018-10-26 | L1Proc-5.6 | GR-20-10-04-gr01 | 5.6 | ||
562264544 | Missing Data. Set to Incomplete and finalized. | 2018-10-27 | L1Proc-5.6 | GR-20-10-04-gr01 | 5.6 | ||
564602913 | Repiped, reprocessed with 5.7, re-exported. | 2018-11-22 | L1Proc-5.6 | GR-20-10-04-gr01 | 5.7 |
Run Number | Reason | Date | FSW | L1Proc | GR version | rePipe | L1Proc |
---|---|---|---|---|---|---|---|
237911263 | silent NFS read error | 2008-07-16 | B1-0-8 | L1Proc-1.63 | Version-v15r39 | ||
237916725 | silent NFS read error | 2008-07-16 | B1-0-8 | L1Proc-1.63 | Version-v15r39 | ||
238008393 | silent NFS read error | 2008-07-17 | B1-0-8 | L1Proc-1.63 | Version-v15r39 | ||
238014123 | silent NFS read error | 2008-07-17 | B1-0-8 | L1Proc-1.61 | Version-v15r24 | ||
238207772 | silent NFS read error | 2008-07-20 | B1-0-8 | L1Proc-1.61 | Version-v15r24 | ||
238759073 | silent NFS read error | 2008-07-26 | B1-0-8 | L1Proc-1.61 | Version-v15r24 | ||
238856333 | silent NFS read error | 2008-07-26 | B1-0-8 | L1Proc-1.61 | Version-v15r24 | ||
239125611 | silent NFS read error | 2008-07-30 | B1-0-8 | L1Proc-1.61 | Version-v15r24 | ||
239129222 | silent NFS read error | 2008-07-30 | B1-0-8 | L1Proc-1.61 | Version-v15r24 | ||
241125159 | 1 missing Datagram | 2008-08-22 | B1-1-0 | L1Proc-1.63 | Version-v15r39 | ||
241491796 | missing chunk in in gcr | 2008-08-27 | B1-1-0 | L1Proc-1.63 | Version-v15r39 | ||
241501554 | 1 missing Datagram | 2008-08-27 | B1-1-0 | L1Proc-1.63 | Version-v15r39 | ||
243170470 | afs cache corruption? | 2008-09-15 | B1-1-0 | L1Proc-1.65 | Version-v15r40 | ||
243197713 | afs cache corruption? | 2008-09-15 | B1-1-0 | L1Proc-1.65 | Version-v15r40 | ||
243307762 | 2 missing Datagrams | 2008-09-17 | B1-1-0 | L1Proc-1.65 | Version-v15r40 | ||
243313738 | 2 missing Datagrams | 2008-09-17 | B1-1-0 | L1Proc-1.65 | Version-v15r40 | ||
243336617 | afs cache corruption? | 2008-09-17 | B1-1-0 | L1Proc-1.65 | Version-v15r40 | ||
243491490 | 4 missing Datagrams | 2008-09-19 | B1-1-0 | L1Proc-1.65 | Version-v15r40 | ||
243873727 | afs cache corruption? | 2008-09-23 | B1-1-0 | L1Proc-1.65 | Version-v15r40 | ||
243926731 | afs cache corruption? | 2008-09-24 | B1-1-0 | L1Proc-1.65 | Version-v15r40 | ||
244143826 | afs cache corruption? | 2008-09-26 | B1-1-0 | L1Proc-1.65 | Version-v15r40 | ||
244167563 | afs cache corruption? | 2008-09-27 | B1-1-0 | L1Proc-1.65 | Version-v15r40 | ||
245667862 | irretrievable packets missing from two datagrams | 2008-10-14 | B1-1-3 | L1Proc-1.67 | Version-v15r47p1 | ||
245673843 | This one was complete, but a retransmission changed its status | 2008-10-14 | B1-1-3 | L1Proc-1.67 | Version-v15r47p1 | ||
245685781 | all packets available; repiping will produce complete data-set | 2008-10-14 | B1-1-3 | L1Proc-1.69 | Version-v15r47p1 | ||
245691461 | all packets available; repiping will produce complete data-set | 2008-10-14 | B1-1-3 | L1Proc-1.67 | Version-v15r47p1 | ||
245697061 | This one was already complete | 2008-10-14 | B1-1-3 | L1Proc-1.67 | Version-v15r47p1 | ||
245702790 | irretrievable packet missing from one datagram | 2008-10-14 | B1-1-3 | L1Proc-1.67 | Version-v15r47p1 | ||
246390295 | Merging problems | 2008-10-22 | B1-1-3 | L1Proc-1.68 | Version-v15r47p1 | ||
246405636 | Merging problems | 2008-10-22 | B1-1-3 | L1Proc-1.68 | Version-v15r47p1 | ||
246447791 | Merging problems | 2008-10-23 | B1-1-3 | L1Proc-1.68 | Version-v15r47p1 | ||
253539991 | Delivery problems ? | 2009-01-13 | B1-1-3 | L1Proc-1.70 | Version-v15r47p1 | ||
260478011 | 1 Missing datagram ??? | 2009-04-03 | B1-1-3 | L1Proc-1.72 | Version-v15r47p7 | ||
261148032 | Insane number of chunks | 2009-04-11 | B1-1-3 | L1Proc-1.72 | Version-v15r47p7 | ||
261154012 | Insane number of chunks | 2009-04-11 | B1-1-3 | L1Proc-1.72 | Version-v15r47p7 | ||
261406015 | 1 Missing datagram ??? | 2009-04-11 | B1-1-3 | L1Proc-1.72 | Version-v15r47p7 | ||
269048825 | Silent staging failures | 2009-07-12 | B1-1-3 | L1Proc-1.73 | Version-v15r47p7 | ||
269054554 | Silent staging failures | 2009-07-12 | B1-1-3 | L1Proc-1.73 | Version-v15r47p7 | ||
269060283 | Silent staging failures | 2009-07-12 | B1-1-3 | L1Proc-1.73 | Version-v15r47p7 | ||
269066012 | Silent staging failures | 2009-07-12 | B1-1-3 | L1Proc-1.73 | Version-v15r47p7 | ||
269071741 | Silent staging failures | 2009-07-12 | B1-1-3 | L1Proc-1.73 | Version-v15r47p7 | ||
269075743 | Silent staging failures | 2009-07-12 | B1-1-3 | L1Proc-1.73 | Version-v15r47p7 | ||
269081879 | Silent staging failures | 2009-07-12 | B1-1-3 | L1Proc-1.73 | Version-v15r47p7 | ||
269087937 | Silent staging failures | 2009-07-12 | B1-1-3 | L1Proc-1.73 | Version-v15r47p7 | ||
269093942 | Silent staging failures | 2009-07-12 | B1-1-3 | L1Proc-1.73 | Version-v15r47p7 | ||
269099929 | Silent staging failures | 2009-07-12 | B1-1-3 | L1Proc-1.73 | Version-v15r47p7 | ||
272627642 | 1 bad event (seg fault) | 2009-08-22 | B1-1-3 | L1Proc-1.75 | Version-v15r47p12 |