...
There are back-out procedures I’ve written just in case there is a need to back out of the pipeline upgrade to 1.5 and use 1.4.2 instead. Those back out procedures would be fine for the case where we turn the pipeline back on and something isn’t immediately fixable, but those back out procedures are not written in the case that we decide to drop back to 1.4.2 after a week. Basically, I keep the snapshot of the tables I’m changing and I can reinsert them for use, but I don’t have a way of updating the tables with any changes.
...
The remaining stages can be boiled down to 3 parts:
Any databases on glast-oracle machines that are not on this list should still be transferred, but they are at the lowest priority.
Date | Checklists | Events |
---|---|---|
5/5, 11am | Readiness meeting | |
5/5, 4pm |
| Shutdown of fastcopy, L1, pipeline. Disable pipeline job submission from FASTCopy. Some web applications may stay up. |
5/5, 5pm |
| Remaining jobs drained out of batch farm. GLASTTREND in read-only. |
5/5, 6pm |
| GLASTTREND transfer (1a) should be be started by now |
5/5, 10pm |
| |
5/6, 6am |
| Transfer of GLASTTREND should be completed. Notification that all applications will be down, including any that may have still been up after 6pm. Transfer of remaining databases (step 2a) on glast-oracle machines should commence. |
5/6, 10am |
| CONTINGENCY POINT: Should the transfer (1a), but NOT conversion (1b), of GLASTTREND exceed this time limit, then we shall ABORT step 2 entirely, and proceed ONLY with step 1. |
5/6, 2pm |
| The conversion of GLASTTREND (1b) must be more than half way finished, and the transfer of the remaining files (2a) must be more than halfway finished. If not, we need to decide on which to proceed with based on how far along they are. |
5/6, 3pm, 3:30pm |
| The transfer of remaining files (2a) must be completed. The conversion of the files (2b) should be started. |
5/6, 5pm |
| The conversion GLASTTREND (1a) must be completed. The conversion of the files (2b) must be completed. GLAST_DP_TEST database must be up, and the schema migrations for the pipeline will be started. |
5/6, 7pm |
| The schema migrations should be finished. GLASTTREND, GLASTGEN, and other databases should be up. Applications should be started. |
...
In Stage 1, we will migrate the Trending data (GLASTTREND account). This involves the following:
Shutdown of fastcopy, L1, pipeline. Disabling of pipeline job submission from FASTCopy automation.
Let remaining jobs drained out of batch farm. GLASTTREND in read-only.
...
...