Here we collect set of instruction for installation of PSDM software at remote sites (site meaning either computer center or your own laptop). The distribution system for software is built on top of the widely accepted software packaging systems - RPM and APT. We provide our own builds for both RPM and APT so we do not depend on the third-party support for them and can install PSDM software on platform where native support for them does not exist. No deep knowledge of RPM or APT is required to use this system, in case any problems please contact SLAC experts.
Currently we support few varieties of the Linux-based OS. OS and its version is designated by a short "platform name". Here is the current list of platforms:
rhel5- Red Hat Enterprise Linux 5.8 or CentOS 5.8.rhel6- Red Hat Enterprise Linux 6.3 or CentOS 6.3.rhel7- Red Hat Enterprise Linux 7.2 or CentOS 7.2.
Different minor versions of the same platform may also work but not be officially supported.
Prerequisites
To install and upgrade both distribution system and PSDM software there is a number of prerequisites which should be resolved first.
Account
It is advisable to run installation from a separate account. Do not use root account for that. Potentially any user account can be used for this but it's better to have a dedicated account just to avoid complications.
Root directory
One has to chose the location where all software is going to be installed. At SLAC everything is installed under directory /reg/g/psdm. For external sites it is better to use the same location as SLAC does to avoid possible bugs due to relocation. If it is not possible to use the same location (e.g. due to lack of permissions) then chose arbitrary location where you can create a new directory for installed software. The directory (called below root directory) has to be created (by root if needed) in advance and be given write permissions for the user account that is going to manage it:
Or if you install it in non-standard location such as your home directory:
Some system packages
PSDM software depends on a small number system packages to be installed by the system itself. When we install our software we do not check (for technical reasons) that those packages are installed, so it is responsibility of the remote site to install these system packages. Below is the list of packages for each supported platform:
Site setup
Once all prerequisites are fulfilled one can move to the next step - setting up software distribution on your site/machine. For that one has to download a script from SLAC and run it, this has to be done from the user account that manages software and has write access to root directory created in previous step.
# for RHEL5/6 curl -o site-setup.sh http://pswww.slac.stanford.edu/psdm-repo/dist_scripts/site-setup.sh # for RHEL7 curl -o site-setup.sh http://pswww.slac.stanford.edu/psdm-repo/dist_scripts/site-setup-rh7.sh # for both chmod 755 site-setup.sh
The script takes one argument which is a location of the root directory:
./site-setup.sh /reg/g/psdm # or `./site-setup.sh $HOME/psdm' if you install it in your home directory
This script downloads RPM and APT and installs them under /reg/g/psdm/sw/dist/apt-rpm and additionally creates few directories necessary for those. When script finishes it will print something like this:
Site setup finished. Please add these lines to .bashrc: ============================================================================ export SIT_ROOT=/reg/g/psdm export PATH=/reg/g/psdm/sw/dist/apt-rpm/rhel6-x86_64/bin:$PATH export APT_CONFIG=/reg/g/psdm/sw/dist/apt-rpm/rhel6-x86_64/etc/apt/apt.conf ============================================================================ (or equivalent to your .cshrc)
If install directory is non-standard then output will look like this:
Site setup finished. Please add these lines to .bashrc: ============================================================================ export SIT_ROOT=/home/psrel/psdm export PATH=/home/psrel/psdm/sw/dist/apt-rpm/rhel5-x86_64/bin:$PATH export APT_CONFIG=/home/psrel/psdm/sw/dist/apt-rpm/rhel5-x86_64/etc/apt/apt.conf export RPMCONFIGDIR=/home/psrel/psdm/sw/dist/apt-rpm/rhel5-x86_64/lib/rpm ============================================================================ (or equivalent to your .cshrc)
Please follow these instructions, running rpm and apt-get commands will need these environment variables. You can also copy and paste these lines into your shell (if you run bash) to continue directly to next steps.
The script will also create .rpmmacros file in your local directory which overrides system-wide settings for rpm. This means that whether you run our newly-installed rpm or system-provided rpm they both will use RPM database that contains PSDM software only. Do not try to install system packages or run yum from the same account (or redefine HOME if you have to).
Release installation
Next step is a first-time installation of the software. You can install release built for your OS only or if you are installing it on a shared filesystem you can install multiple OS at once. Do not forget to set environment variable that `site-setup.sh` script printed.
For RHEL6 or CentOS6:
apt-get update apt-get install psdm-release-ana-latest-rhel6
For RHEL5 or CentOS5:
apt-get update apt-get install psdm-release-ana-latest-rhel5
More than one platform can be installed at once, e.g.:
apt-get update apt-get install psdm-release-ana-latest-rhel5 psdm-release-ana-latest-rhel6
These commands will download latest analysis release and all necessary packages and install them under /reg/g/psdm/sw/releases and /reg/g/psdm/sw/external.
After this it is also a good idea to create an "ana-current" link in /reg/g/psdm/sw/releases, some scripts work better if this link exists, in particular /reg/g/psdm/etc/ana_env.*sh scripts need this link. To create it you need to know which release you have just downloaded. To determine this look into /reg/g/psdm/sw/releases directory or in the output of the above apt-get command. Once you know release name run this command:
/reg/g/psdm/bin/relcurrent ana-X.Y.Z # or $HOME/psdm/bin/relcurrent ana-X.Y.Z
where ana-X.Y.Z should be replaced with the real release name.
This is it! Source the script
. /reg/g/psdm/etc/ana_env.sh # or . $HOME/psdm/etc/ana_env.sh
and you are ready. (You should probably do this from some other account).
Installation Customization
If you have installed a release in a non-standard locations (rather than at /reg/g/psdm) at your site/computer then the MPI application won't be able to find certain files. That's because the installation path for the library is hard-wired into OpenMPI binaries and libraries at a time when the library is being built. This problem is explained in this OpenMPI FAQ. A solution to this problem is to find a location where the library is installed within your release and set an environment variable OPAL_PREFIX overriding the base path. Here is how this variable could be set at SLAC for RHEL5 if you're using TCSH.
setenv OPAL_PREFIX /reg/g/psdm/sw/external/openmpi/1.8.1/x86_64-rhel5-gcc41-opt/
Also, starting with release ana-0.14.1 you can specify the location of your data directory (the equivalent of "/reg/d/psdm" at SLAC) with the environment variable SIT_PSDM_DATA. You can also do this when you specify the DataSource with an option similar to "dir=/reg/d/ffb/amo/amoi0114/xtc", but the environment variable provides a clean method to do so "globally".
MPI
Starting with ana-0.18.0, the openmpi package is built to dynamically load lsf components. Offsite users will typically not have lsf. sit_setup will, by default, set the following environment variables
OMPI_MCA_ess ^lsf
OMPI_MCA_ras ^lsf
OMPI_MCA_plm ^lsf
Advanced users may want to control these environment variables themselves. If so, set
SIT_NO_MODIFY_OMPI_ENV=1
If your offsite has a lsf installation you want to try, set LSF_PACKAGE_LIB appropriately. sit_setup will use this directory instead.
Release upgrade
When you want to upgrade to the new version of the analysis release run this from the same account that was used to install everything:
apt-get update apt-get dist-upgrade
Note that if you install software in non-standard location the `SIT_ROOT` environment variable must be set before you run these commands.
These commands will find out what is the most recent release available at SLAC, download it with all dependencies and install. If you installed release for multiple architectures (rhel5/rhel6) then they all will be upgraded.
Once you have tested this new release and happy with it you can point ana-current to this new release, just run same relcurrent command again but give it new release name.
Overview
Content Tools