Page History

...

• Python 2.7.4 32-bit (physics default)
• Python 2.7.13 64-bit (softegr default)
  

The currently installed package list for Python 2.7.13 is here.

Base Python Installation

The steps to install vanilla Python 2.7.x in LCLS production are as follows (examples use Python 2.7.13):

• From an internet-accessible machine, download the latest Python 2.7.13 source tarball (https://www.python.org/downloads/release/python-2713/)

• Create the Python installation directory structure in LCLS production: $PACKAGE_TOP/python/python<version> (e.g., $PACKAGE_TOP/python/python2.7.13), which should look as follows:

  Code Block
  language bash theme Midnight
  $ ls /usr/local/lcls/package/python/python2.7.13/linux-x86_64 bin include lib man share src

• Transfer the Python source tarball to $PACKAGE_TOP/python/python<version>/src:
  

  Code Block
  language bash theme Midnight
  $ ls /usr/local/lcls/package/python/python2.7.13/linux-x86_64/src Python-2.7.13.tgz

• Configure the build with the following options:

  Anchor
  bootstrap_pip bootstrap_pip

  Info
  --prefix=$PACKAGE_TOP/python/python2.7.13/linux-x86_64 (To create all build directories under the current structure) --enable-shared (To allow building of shared libraries [.so files], required for packages such as PyQt) --with-ensurepip=upgrade (To bootstrap the latest pip installer for Python 2.7.x >= 2.7.9)

  Code Block
  language bash theme Midnight
  $ ./configure --prefix=$PACKAGE_TOP/python/python2.7.13/linux-x86_64 --enable-shared --with-ensurepip=upgrade $ make $ make install

• Test the installation by setting the environment to use the newly-installed Python instance (example uses a script):

  Code Block
  language bash theme Midnight
  $ cat init_python2.7.13.bash #!/bin/bash # Example of Setssetting the user environment to use an instance of Python2.7.13 if [ -d /afs/slac/g/lcls ]; then export LCLS_ROOT=/afs/slac/g/lcls else export LCLS_ROOT=/usr/local/lcls fi export ORACLE_HOME=${LCLS_ROOT}/package/oracle/product/11.2.0.4/linux-x86_64 export QT_HOME=${LCLS_ROOT}/package/Qt/Qt-5.9.1/linux-x86_64 export PYTHON_ROOT=${LCLS_ROOT}/package/python/python2.7.13/linux-x86_64 export PATH=${QT_HOME}/bin:${ORACLE_HOME}/bin:${PYTHON_ROOT}/bin:${PATH} export LD_LIBRARY_PATH=${EPICS_BASE_RELEASE}/lib/linux-x86_64:${EPICS_EXTENSIONS}/lib/linux-x86_64:${EPICS_BASE_TOP}/base-cpp-R4-6-0/lib/linux-x86_64:${QT_HOME}/lib:${ORACLE_HOME}/lib:${PYTHON_ROOT}/lib:${PYTHON_ROOT}/build/Python2.7.13/Lib:${LD_LIBRARY_PATH} $ source init_python2.7.13.bash

  
  

• Check that the core installation includes pip and setuptools (required for building Python modules from source):

  Code Block
  language bash theme Midnight
  $ pip list pip (9.0.1) setuptools (28.8.0)

...

• Core packages: packages for installing and building Python packages from sources or binaries (e.g. pip, setuptools, wheel) [$python_home/src/core-packages]
• Site packages: packages obtained via the official Python Package Index (PyPI) repository ($python_home/src/site-packages)
• External packages: packages obtained from non-PyPI sources, such as GitHub or SourceForge ($python_home/src/external-packages)
• SLAC packages: packages created and/or maintained by SLAC entities ($python_home/src/slac-packages)

To install packages, packages:

• From an internet-accessible machine, download the appropriate source tarballs (when available, otherwise omit the "--no-binary :all:" argument) from PyPI using pip:

  Code Block
  language bash theme Midnight
  # Example: # Download source tarballs of all modules listed in requirements file to the destination directory $ pip download --dest=<download_directory> --no-binary :all: -r installed-modules.txt # Download source tarball of a specific module to the destination directory $ pip download --dest=<download_directory> --no-binary :all: six

• Copy the source files over to the appropriate directory in the Python 2.7.13 deployment tree (/usr/local/lcls/package/python/python2.7.13/linux-x86_64/src) depending on package origin (see below).
• run `pip install` and define the paths to the package source files with the --find-links option (note that pip will make a best effort to

...

• satisfy dependencies, but this is by no means guaranteed. The --find-links argument can be specified more than once to look for files in multiple locations. Barring this, the --no-deps argument can be used to ignore dependencies):

# pip install --no-cache-dir --no-index --find-links=/path/to/module/source/dir [module_source|-r requirements.txt]
#
# Examples:
# Install using requirements file
$ pip install --no-cache-dir --no-index --find-links=/usr/local/lcls/package/python/python2.7.13/linux-x86_64/src/site-packages -r site-requirements.txt

# Install specific module (requires module tarball/wheel/egg file to be located in the directory specified by --find-links argument. If multiple packages
# match the name, the tarball/wheel/egg with the latest version number will be used.)
$ pip install --no-cache-dir --no-index --find-links=/usr/local/lcls/package/python/python2.7.13/linux-x86_64/src/site-packages six

# Install module from specific source file
$ pip install --no-cache-dir --no-index --find-links=/usr/local/lcls/package/python/python2.7.13/linux-x86_64/src/slac-packages pydm-1.11.0.tar.gz

Core packages

Core packages are required in order to install and build other Python packages from source. The pip and setuptools packages should be bootstrapped via the base Python installation process (see above). The wheels package is required to install packages from wheels (.whl), which is necessary if the source tarball cannot be obtained.

...

• mps_tools: https://github.com/slaclab/mps_tools
• mpsbypass: https://github.com/slaclab/mpsbypass
• pyca: https://github.com/slaclab/pyca
• pydm: https://github.com/slaclab/pydm
• slacepics: https://github.com/slaclab/slacepics
• subprocessca: https://github.com/slaclab/subprocessca
• TimeChart: https://github.com/slaclab/timechart

...

• PIL: http://pythonware.com/products/pil/
• PyQwt: https://github.com/PyQwtSIP: https://www.riverbankcomputing.com/software/sip/download

$ cat /usr/local/lcls/package/python/python2.7.13/linux-x86_64/src/external-packages/external-requirements.txt
PIL
PyQwt

...

• Set the QT_HOME environment variable to the location of the Qt5 deployment in the LCLS Controls Environment:

  Code Block
  language bash theme Midnight
  $ export QT_HOME=/usr/local/lcls/package/Qt/Qt-5.9.1/linux-x86_64

• Ensure that the Python 2.7.13 deployment to build and install PyQt5 and SIP against is set as the default Python instance:

  Code Block
  language bash theme Midnight
  $ export PATH=/usr/local/lcls/package/python/python2.7.13/linux-x86_64/bin:$PATH $ export LD_LIBRARY_PATH=/usr/local/lcls/package/python/python2.7.13/linux-x86_64/lib:/usr/local/lcls/package/python/python2.7.13/linux-x86_64/lib:/usr/local/lcls/package/python/python2.7.13/linux-x86_64/lib/python2.7/lib-dynload:/usr/local/lcls/package/python/python2.7.13/linux-x86_64/build/Python2.7.13/Lib:$LD_LIBRARY_PATH # test $ python --version Python 2.7.13

• Extract and build the SIP module, then install it for the Python 2.7.13 instance:

  Code Block
  language bash theme Midnight
  $ cd /usr/local/lcls/package/python/python2.7.13/linux-x86_64/src/external-packages $ tar -xzvf sip-4.19.6.tar.gz $ cd sip-4.19.6 $ python configure.py $ make $ make install

• Set a proper gcc build chain:

  Code Block
  language bash theme Midnight
  $ export PATH=/usr/local/lcls/package/gcc/gcc-4.9.4/bin:$PATH $ export LD_LIBRARY_PATH=/usr/local/lcls/package/gcc/gcc-4.9.4/linux-x86_64/lib:/usr/local/lcls/package/gcc/gcc-4.9.4/linux-x86_64/lib64:$LD_LIBRARY_PATH

• Extract and build the PyQt5 module, then install it for the Python 2.7.13 intstance:

  Code Block
  language bash theme Midnight
  $ cd /usr/local/lcls/package/python/python2.7.13/linux-x86_64/src/external-packages $ tar -xzvf PyQt5_gpl-5.9.2.tar.gz $ cd PyQt5_gpl-5.9.2 $ python configure.py $ make $ make install

References:

• Python 2.7 (deprecated end-of-life as of January 2020): https://docs.python.org/2/
• PyPI - The Python Package Index: https://pypi.python.org/pypi
• pip - The Python Package Authority's preferred tool for installing packages: https://pip.pypa.io/en/stable/user_guide/#installing-packages
• Jupyter: https://jupyter.org
• PyQt: https://www.riverbankcomputing.com/software/pyqt/intro
• SIP: https://www.riverbankcomputing.com/software/sip/download
• Scikit-learn: http://scikit-learn.org/stable/index.html

...