Versions Compared

Old Version 3

changes.mady.by.user Brian Van Klaveren

Saved on

compared with

New Version 4

changes.mady.by.user Brian Van Klaveren

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  1.  Migration of GLASTTREND
    1. Transfer of files
    2. Conversion of files
    3. Starting of the database
  2. Migration of GLAST_DP_TEST, GLASTGEN, etc...
    1. Transfer of files
    2. Conversion of files
    3. Starting of the database

Timeline

DateEvents
5/5, 11amReadiness

...

meeting
5/5, 4pm

...

...

Shutdown of fastcopy, L1, pipeline. Some web applications may stay up.
5/5, 5pmRemaining jobs drained out of batch farm. GLASTTREND in read-only.
5/5, 6pmGLASTTREND transfer (1a) should be be started by now
5/5, 10pm

...

OPTIONAL:

Once at least 600GB has been transferred, we might be able to start conversion of some GLASTTREND files.

Not too likely to impact transfer performance, as it’s not likely to be hitting the same discs.

5/6, 6am

Transfer of GLASTTREND should be completed.

Notification that all applications will be down, including any that may have still been up after 6pm.

Transfer of remaining databases (step 2a) on glast-oracle machines should commence.

5/6, 10am

...

CONTINGENCY POINT:

Should the transfer (1a), but NOT conversion (1b), of GLASTTREND exceed this time limit, then we shall ABORT step 2 entirely, and proceed ONLY with step 1.

5/6, 2pm

The conversion of GLASTTREND (1b) must be more than half way finished, and

...

the transfer of the remaining files (2a) must be more than halfway finished.

If not, we need to decide on which to proceed with based on how far along they are.

5/6, 3pm

The transfer of remaining files (2a) must be completed.

The conversion of the files (2b) should be started.

5/6, 5pm

The conversion GLASTTREND (1a) must be completed.

The conversion of the files (2b) must be completed. GLAST_DP_TEST database must be up, and the schema migrations for the pipeline will be started.

5/6, 7pm

The schema migrations should be finished.

GLASTTREND, GLASTGEN, and other databases should be up.

Applications should be started.

 

Stages with detailed steps

There was actually 3 stages, the first stage was completed Wednesday, April 30. That stage was to migrate ISOC_FLIGHT tablespace.

...

  1. Take a full outage. Everything which was still connecting to glast-oracle03/04 will be shut down.
  2. Ian will start transferring the rest of the database files for all other accounts. This includes:
    1. GLAST_DP_TEST
    2. CONFIG (GLASTGEN)
    3. GLAST_ASP
    4. GLAST_RSP
    5. GLAST_ISOC
    6. ... and various other accounts.
  3. Connections to the databases in tnsnames.ora will be updated
  4. GLAST_DP_TEST will be prioritized to be transferred first.
  5. Headers will be modified for little-endianness on files which have finished transferring while other files are transferring.
  6. Upon successful transfer, which will take an estimated 4 hours:
    1. Ian will create 4 new database files to add to available space for a schema change
    2. Ian verifies database configuration (file location, permissions), the database is brought up
    3. The schema change described on the Pipeline Migration page will be performed
  7. Services will be started back up
    1. All tomcat servers should be restarted
    2. Any applications that were left running which rely on database connectivity should be restarted