• Created by Unknown User (davidsch), last modified on Dec 12, 2016

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 3 Next »

Introduction

This page covers documentation about the conda release system for LCLS users.

Users need to do the following

  • use bash as your shell, not tcsh or csh (conda doesn't support csh)
  • run the command: source /reg/g/psdm/etc/ana_env.sh as usual (explained in the psana python setup example)
  • run the command: source conda_setup

 Sourcing the conda_setup script does a few things

  • modifies certain linux environment variables that can affect conda
  • identifies the most recent conda environment built for LCLS data analysis
  • activates that conda environment

At this point your linux prompt should change to something that starts with the name of the conda environment. For example,

(ana-1.0.7)

Python 3

The LCLS Data Analysis group will maintain conda environments with python 3. To activate the latest python 3 environment, pass the --py3 flag to conda_setup. That is run

source conda_setup --py3

Note - psana has not been ported to python 3. The python 3 environments do not include software to work with original LCLS experiment data files recorded by the data acquisition system (however you can work with translated hdf5 files).

Notes on Provided Conda Environments

  • source conda_setup -h for full help on the script
  • after sourcing conda_setup, you can execute the undo_conda command to restore your environment
  • see conda ana release notes for release notes on these environments
  • to switch to an older environment called name, you can use conda_setup with the --env name arguments, or use the standard conda source activate name command.

Newer Package Versions

The conda environments will keep up with the latest versions for many standard python packages. Some notable version changes from the RPM based releases (as of December 2016):

  • ipython, version 2.3 -> 5.1
  • matplotlib, version 1.4.3 -> 1.5.1
  • openmpi, version 1.8.3 -> 1.10.2
  • mpi4py, version 1.3.1 -> 2.0.0
  • numpy, version 1.9.2 -> 1.11.2 also conda version is based on the intel MK

New package versions means newer features, bug fixes, but it can also mean old interfaces are deprecated, and new bugs crop in. Some users may need to update their code to run with later versions of packages.

We do not use the latest version of packages that brake psana. For instance, we do not use h5py 2.6.0 since it cannot read the hdf5 files produced by the Translator. We use h5py 2.5.

Red Hat 5, 6, 7 Issues

Most users will run on rhel7 machines - anyone doing analysis on the interactive machines or submitting batch jobs to the usual queues will use rhel7 machines. There are some rhel6 and rhel5 machines in the experiment hutch control rooms. In particular users running from shared memory may have to run on a rhel6 or rhel5 host.

Continuum (which produced conda) suggests linux packages be built once in a way that runs on any linux operating system - rhel5, rhel6, rhel7, ubuntu, etc. In practice, for large packages like openmpi that the LCLS Data Analysis group must build local versions of, this can get very technical and lead to a brittle system. Similarly, conda-forge, a primary source of conda packages, decided to not support rhel5 for technical reasons.

The LCLS Data Analysis group is presently maintaining 3 separate conda installations, one for rhel5, rhel6, and rhel7. Locally built packages, like openmpi, hdf5, and psana, will be built natively on each platform. The conda_setup script will automatically detect rhel5 vs rhel6 vs rhel7 and activate an environment in the appropriate installation. Use conda_setup to get into a conda environment.

However since conda-forge does not support rhel5, certain packages will not function on rhel5 (for example, opencv, which we obtain from conda-forge). In general, our support for rhel5 is more limited with the conda releases than the RPM releases. Rhel5 users may not be able to use a conda environment.

Moreover it is possible that we install a package that is limited to rhel7. Today - the only such package is tensorflow.

New Packages and Updates

It is much easier to add new packages to the LCLS conda environments than the RPM releases. Please contact us - pcds-ana-l@slac.stanford.edu for requests. Packages that are:

  1. generally useful
  2. come from standard conda channels like continuum defaults or conda-forge
  3. do not introduce dependencies that interfere with psana

Will be straightforward to install.

Updating existing packages is straightforward - however we will always build new conda environments with a bumped up version number before changing an existing package.

User Conda Environments

A great way to experiment with a new package is to create your own conda environment and install the package there (this is similar to using virtualenv). This will work fine for environments based on packages from continuum, conda-forge, etc. However if you want to create your own conda environment that includes LCLS software such as psana, or our local build of openmpi or hdf5 there will be issues. We recommend you look the Psana Developer Documentation to see if a psana test release will fit your needs. If not, here are some issues:

  • You cannot use your environment across rhel5, rhel6, and rhel7. All conda environments live in one place. You could create three environments, a myenv-rhel5, myenv-rhel6 and myenv-rhel7, however presently we do not include  rhel5/rhel6/rhel7 in our psana conda package names, consequently your three environments will all link to the same build of psana.
  • There are issues with the conda create --clone command (a natural way to extend an environment like ana-1.0.7), for instance, mpirun does not seem to be installed properly in the cloned environment
  • You need to define the environment variables SIT_ROOT and SIT_DATA for psana (look at the file /reg/g/psdm/sw/conda/inst/miniconda2-prod-rhel7/envs/ana-1.0.7/etc/conda/activate.d/env_vars.sh for an example of how).
  • The biggest issue, is that all 3GB of software in the LCLS conda environment will get copied into your home directory.

Supporting softlinks from user home directories to a central package repository is a feature under development at continuum. Before users can create their own conda environment with psana that they run cross platform, we will need to do some development.

  • No labels