...
Just like the CMT Release Manager, the SCons Release Manager is split into several components. These components are the batch submission system, the workflow system, and the release manager system. Unlike the CMT Release Manager, the SCons Release Manager is not written in perl. It is written in C++ relying mostly on the Qt library. Additionally, it depends on the Boost Spirit library, and the stx-exparser library. The Qt library is used for most platform independent interactions. The Boost spirit and stx-exparser are used on the linux side only for linux components only (in particular, for a Workflow application only).
Qt
A major component of the SCons infrastructure is the Qt library. For linux and the mac the source code is obtained from the trolltech website and compiled. In both cases it's compiled as a static library. For windows the installation is a bit more complicated. The windows installation doesn't come with the mysql plugins pre-compiled. As a result it needs to be compiled after installing Qt for windows. From the trolltech webpage one needs to download the LGPL/Free version for windows. There are several choices to download. The appropriate one is the Framework Only version for Windows. This version is complied with mingw so the machine it's installed on will need mingw. The installer for Qt will take care of installing it but it should NOT be installed on any of the machines that are used by the ReleaseManager as SCons might get confused with mingw and visual c++ both installed.
...
The Batch Submission System consists of two applications. The lsf Daemon and the lsf callback application. The daemon is setup to run on fermilnx-v03 (a virtual machine) via trscron (trscron is used instead of regular cron because the LSF processes need AFS tokens and normal cron doesn't provide them). The daemon runs for an hour and is then retriggered.
It checks the database for new jobs. Jobs are submitted to lsf LSF in suspend mode. For each job, there are three lsf LSF tasks submitted. The first tasks task is the actual command to be executed. The second task is a call to the lsf callback application with the condition that it be executed as soon as the first task starts execution. The third task is a call to the lsf callback application with the condition that it be executed as soon as the first task finishes execution. All lsf LSF tasks are submitted and initially put into suspend mode. If all three tasks are submitted successfully, the jobs are changed from suspend to pending mode so they actually start execution. If at any time anything goes wrong, the lsf LSF tasks are killed and the run in the table is marked as failed. The job is put back into the queue to be attempted at a later time as a new run.
The lsf callback application is called once when the lsf LSF task starts and once when it finishes. When started, it updates the run table with the information about the start time. Additionally, it checkes the callback table to determine if the higher layers wish to be notified when the job has started. If so, it calls the callback command to notify the higher levels. When called for a task that has finished. The callback system checks the lsf LSF output, parses some of the information, like the return code, and stores that information in the run, job, and output tables. Additionally, it checks for any callback information for the higher layers.
...
- build – This contains the platform dependent information for a particular build.
- buildPackage – This contains the platform independent information for a particular build.
- deletedBuild – This table contains has same structure as the build table but contains the entries for builds that have been completely removed by the eraseBuild program.
- entry - this table was migraged from glastdb and hold the log entries from the operation of all the RM processes. Logs are kept for 5 days and then removed to prevent the database from filling up.
- extLib – This contains the external libraries used for a particular build.
- os – This contains a list of operating systems currently supported.
- outputMessage – This contains the checkout or compile output for a particular build.
- package – This contains a list of checkout packages currently supported.
- settings – This contains both global and build specific settings for the Release Manager.
- subPackage – This contains a list of subPackages and their compile status for a particular build.
- subPackageCompileFailures – This contains the location of specific compile failures in the compile output for a particular subPackage.
- variant – This contains a list of supported variants.
- versionType – This contains a list of supported versionTypes.
...
To re-enable tagging, just undo the changes made above.
Miscellaneous Details
This section provides links to various sub-pages describing random bits of SCons lore.