This document is meant to be a guide for migration and testing now that a snapshot of the RM databases has been captured and installed on mysql-node01, which has MySQL server 5.5.

CMT RM Web Pages

Is it possible to make alternate cgi scripts which use as host mysql-node01 rather than glastDB? If so, we can test all functionality without any impact on production pages. 

I think I found the location of the scripts which actually get called.  It's highly protected, but copies of the files are available elsewhere for browsing.  If I understand correctly, the production files also ultimately use the information in DBConf to connect, but it doesn't look too hard to make slightly-modified versions with different names which would use an alternate DBConf, pointing to mysql-node1 (and no doubt easier still for someone with experience with cgi scripts). We'd probably need help from unix-admin to install the modified scripts in the official location.

Update: unix-admin can install alternate cgi scripts to be invoked by an alternate SLAC-internal-only test web server.  I've created the alternates and informed unix-admin they're ready to go.

Update to update: The alternate pages can be viewed (internal SLAC network only) at, e.g., http://www-dev.slac.stanford.edu/cgi-wrap/ReleaseManager

Table Naming and Structure

(warning) In the Workflow database there is a table named trigger.  As of MySQL 5.0 this is a reserved word.  It may still be used as a table name, but must be quoted with back-ticks like this;  `trigger`

So far I've only tested this with the command-line client program mysql. There is one Perl module (Workflow.pm) which probably needs to be modified in a couple places.  I haven't been able to find any other references to this table.

(warning) There was a similar problem with the newly reserved word condition which is used as a column name in one of the tables.  The only references to it in the Perl scripts are also in Workflow.pm.

The results of the describe <tablename> command are not always the same between 4.1 and 5.5, even if show create table <tablename> has identical output for both.  In particular, the describe outputs differ for columns created NOT NULL.  Probably RM never uses the describe command, or if it does, does not look specifically for NOT NULL columns, but we should confirm.  There are a couple other small differences in describe output between old and new MySQL versions which are even less likely to be significant (in values displayed under headings "Key" and "Default").

Instances of Use of MySQL by the CMT RM

All over the place.  We decided not to pursue testing more individual pieces (other than web pages) since we had reason to believe we were close and there is no user activity to speak of this week. 

Use of  MySQL 4.1 Client Library

The CMT RM Perl scripts uses the MySQL client library indirectly via the Perl module PBI, and the MySQL driver DBD::mysql.  On Linux everything we require is already installed and on rhel5 or rhel6 machines will be using MySQL 5.0 or newer client library.  On Windows we're currently using a MySQL 4.1.22 client which at some point will need to be upgraded, but not urgently (should still work with the newer server).

~glastrm/mysql-proxy might be relevant.

Testing steps

Just a proposal; by no means the final word!

  1. Make copies of cgi scripts used for web pages which point to copied db on mysql-node01. Make an alternate module to DBConf.pm which points to mysql-node01 (but leave the original intact). Verify that all functions still work. (tick) All ReleaseManager pages have been cloned and directed to mysql-node01; they seem ok.
  2. Compare glastDB and mysql-node1 tables via command-line mysql client program. (tick) (warning) Done. There is one potentially troublesome issue: use of reserved word trigger as table name.
  3. As much as possible, isolate parts of CMT RM which access db and test.  That is, make alternate scripts which point at the alternate  DBConf.pm and run them by hand. Other than confirming with test script that quoting trigger in Perl usage would satisfy MySQL 5.5, did not pursue this.
  4. Fix any problems found up to this point. Changed a few lines in Workflow.pm.(tick)
  5. Make a copy of installer script(s) pointed to mysql-node01 and test.  (tick)
  6. Full system test of the RM with mysql-node01.  (tick) That is
    1. warn everyone not connected with testing to avoid any activity which would involve writing to db (e.g., tagging)
    2. shut of CMT RM
    3. save original DBConf.pm; rename alternate to DBConf.pm. If there are any other sources of host name besides DBConf.pm adjust those as well.
    4. Turn CMT RM on.  Verify functionality (LATEST tag, release tag, etc.)
    5. Turn off CMT RM again; point everything back to glastDB; turn back on.
  7.   The actual move:
    1. send general warning.  There should be no tagging during the move at the very least.
    2. shut off RM.
    3. dump databases on glastDB, refresh copies on mysql-node01
    4. point glastDB alias to mysql-node01  (did we receive confirmation that we will retain the glastDB alias?)
    5. restart everything; issue all-clear

Reference

Perl and MySQL

http://search.cpan.org/~capttofu/DBD-mysql-3.0002/lib/DBD/mysql.pm

CMT RM Script Inventory

Cron system

The following are all mentioned in the crontab table and can be found in ~glast/infraCron:

batchSub.pl
cleanSessions.pl
createLastest.pl
logClean.pl
rmTodo.pl
workflow.pl

The same directory contains the script cron.pl

  • No labels