Initial Goal
..is to run Gleam in a VM. Probably that entails
- Make an "appliance": rhel5 or rhel6 VM with Gleam executable and all required libraries, job options, etc.
To start, would be simplest to run MC job, write output to VM local disk.
- Write program or script for host machine which will do one of the following:
- start up Gleam in the already-up-and-running VM
- first boot up the VM, then ask it to run Gleam
Issues and choices
For the most part the following sections concern decisions for the longer term, not just for the initial goal of running Gleam in a VM, in fact some of them are irrelevant except in the longer term.
Which virtualization software?
I'm experimenting with VirtualBox to start, but since we're only concerned with Linux for now VMware (VMplayer product) would probably serve; there may be other suitable candidates. Advantages of VirtualBox compared to VMware are
- Supported OSes
- VirtualBox supports Linux, Windows and Mac for client and host (though there are a bunch of restrictions for Mac, mostly due to the way Apple does things).
- VMplayer supports Linux and Windows
- API
- VirtualBox API (the same one used internally, hence complete) supports C++, python, and any other languages that understand (Microsoft) COM or, on non-Windows platforms, XPCOM for applications running on the host. There is also a web service which implements nearly the complete API. Any language with a toolkit for wsdl can use this. It's especially easy to use from Python and Java because the VirtualBox SDK includes wrapper classes for these languages.
- The free SDK product VIX can be used with VMplayer. Functionality is somewhat limited compared to use with non-free products VMworkstation and vSphere, but may be sufficient for us. In particular, it does include the ability to start a program on a VM. Bindings exist only for C, Perl, and (Microsoft) COM.
Functions to be provided on VMs
- Run batch (pipeline) jobs using Gleam, both for MC generation and for real data reconstruction
- Build Gleam and related software (need access to legacy compiler, external libs) e.g. via Release Manager
- Provide interactive development environment for Gleam programmers (interactive login, editor, debugger, etc.).
- Allow for creation and installation of new version of VM including, for example, new build of Gleam
What should a batch job do?
This will depend on the state of the VM before the job is run. From the VirtualBox API one could in principle create the VM, configure it, boot it and ask it to run something all from the API, but for our use we'll want to, at the very least, create, configure and register the VM beforehand. Booting a VM for every job start does of course entail some overhead (SL6 takes a couple minutes to boot up on my 2.26 GHz dual core laptop, running rhel5), but that may be tolerable in the context of our typical multi-hour batch job. Alternatively, if the (presumably SCCS-maintained) host machine is dedicated to our use, some number of guest VMs can be up and running at all times. In that case we'd want to monitor, to be sure they really are up and usable, and ideally have an automated procedure to reboot any crashed or hung VMs.
Resource access and security
For our intended uses VMs must have input from the outside world and must be able to write output somewhere which is accessible to non-VMs, however the VM is perhaps running an unpatched legacy OS with known security holes.
There is no need for the VMs to be general purpose machines. Restrictions like the following should keep the machines (and other SLAC resources) from being trashed without interfering with the functions we require.
- VMs should not have write access to any "regular" public space (space to which centrally-supported public log-in machines also have access).
- VMs need have no access at all to regular user home directories.