...
- From an internet-accessible machine, download the latest Python 2.7.x source tarball (https://www.python.org/downloads/source/)
Create the Python installation directory structure in LCLS production: $PACKAGE_TOP/python/python<version> (e.g., $PACKAGE_TOP/python/python2.7.13),
which should look as follows:Code Block language bash theme Midnight $ ls /usr/local/lcls/package/python/python2.7.13/linux-x86_64 bin include lib man share src
Transfer the Python source tarball to $PACKAGE_TOP/python/python<version>/src:
Code Block language bash theme Midnight $ ls /usr/local/lcls/package/python/python2.7.13/linux-x86_64/src Python-2.7.13.tgz
Configure the build with the following options:
Anchor bootstrap_pip bootstrap_pip Info --prefix=$PACKAGE_TOP/python/python2.7.13/linux-x86_64 (To create all build directories under the current structure)
--enable-shared (To allow building of shared libraries [.so files], required for packages such as PyQt)
--with-ensurepip=upgrade (To bootstrap the latest pip installer for Python 2.7.x >= 2.7.9)Code Block language bash theme Midnight $ ./configure --prefix=$PACKAGE_TOP/python/python2.7.13/linux-x86_64 --enable-shared --with-ensurepip=upgrade $ make $ make install
Set the environment to use the newly-installed Python instance (example uses a script):
Code Block language bash theme Midnight $ cat init_python2.7.13.bash #!/bin/bash # Sets the user environment to use Python2.7.13 if [ -d /afs/slac/g/lcls ]; then export LCLS_ROOT=/afs/slac/g/lcls else export LCLS_ROOT=/usr/local/lcls fi export ORACLE_HOME=${LCLS_ROOT}/package/oracle/product/11.2.0.4/linux-x86_64 export QT_HOME=${LCLS_ROOT}/package/Qt/Qt-5.9.1/linux-x86_64 export PYTHON_ROOT=${LCLS_ROOT}/package/python/python2.7.13/linux-x86_64 export PATH=${QT_HOME}/bin:${ORACLE_HOME}/bin:${PYTHON_ROOT}/bin:${PATH} export LD_LIBRARY_PATH=${EPICS_BASE_RELEASE}/lib/linux-x86_64:${EPICS_EXTENSIONS}/lib/linux-x86_64:${EPICS_BASE_TOP}/base-cpp-R4-6-0/lib/linux-x86_64:${QT_HOME}/lib:${ORACLE_HOME}/lib:${PYTHON_ROOT}/lib:${PYTHON_ROOT}/build/Python2.7.13/Lib:${LD_LIBRARY_PATH} $ source init_python2.7.13.bash
Check that the core installation includes pip and setuptools (required for building Python modules from source):
Code Block language bash theme Midnight $ pip list pip (9.0.1) setuptools (28.8.0)
...
Python packages installed in LCLS controls production are obtained from various sources. When possible, it is preferable to build the packages from source in the production environment to ensure compatibility.
...
satisfy dependencies, but this is by no means guaranteed. The --find-links argument can be specified more than once to look for files in multiple locations. Barring this, the --no-deps argument can be used to ignore dependencies):
...
Core packages are required in order to install and build other Python packages from source. The pip and setuptools packages should be bootstrapped via the base Python installation process (see above). The wheels package is required to install packages from wheels (.whl), which is necessary if the source tarball cannot be obtained.
...
Site packages are available on the official Python Package Index (PyPI) repository. When possible, packages should be obtained from this repo for maintainability and consistency. Python packages with complex dependencies have their own requirements files that are referenced by the top-level site-requirements.txt file. Note that redundant packages are ignored in later requirements files if the dependency is already satisfied:
Flask
Code Block language bash theme Midnight $ cat /usr/local/lcls/package/python/python2.7.13/linux-x86_64/src/site-packages/Flask-requirements.txt Flask Werkzeug Jinja2 itsdangerous click MarkupSafe
IPython
Code Block language bash theme Midnight $ cat /usr/local/lcls/package/python/python2.7.13/linux-x86_64/src/site-packages/ipython-requirements.txt appdirs backports.shutil-get-terminal-size decorator enum34 ipython ipython-genutils packaging pathlib2 pexpect pickleshare prompt-toolkit ptyprocess pygments pyparsing scandir setuptools simplegeneric six traitlets wcwidth
Jupyter
Code Block language bash theme Midnight $ cat /usr/local/lcls/package/python/python2.7.13/linux-x86_64/src/site-packages/jupyter-requirements.txt alabaster appdirs Babel backports-abc backports.shutil-get-terminal-size backports.ssl-match-hostname bleach certifi Cheetah click configparser cx-Oracle cycler decorator docutils dpkt entrypoints enum34 first Flask functools32 html5lib imagesize ipykernel ipython ipython-genutils ipywidgets itsdangerous Jinja2 jsonschema jupyter jupyter-client jupyter-console jupyter-core Markdown MarkupSafe matplotlib mistune mpmath nbconvert nbformat nose notebook numpy packaging pandas pandocfilters pathlib2 pexpect pickleshare PIL pip pip-tools plex prompt-toolkit ptyprocess Pygments pyparsing pyqtgraph PyQwt python-dateutil pytz pyzmq qtconsole requests roman scandir scikit-learn scipy setuptools setuptools-scm simplegeneric singledispatch six snowballstemmer Sphinx subprocess32 sympy terminado testpath tornado traitlets virtualenv wcwidth webencodings Werkzeug wheel widgetsnbextension
pip-tools
Code Block language bash theme Midnight $ cat /usr/local/lcls/package/python/python2.7.13/linux-x86_64/src/site-packages/pip-tools-requirements.txt pip-tools click first setuptools-scm six
Scikit-learn
Code Block language bash theme Midnight $ cat /usr/local/lcls/package/python/python2.7.13/linux-x86_64/src/site-packages/scikit-learn-requirements.txt numpy scipy scikit-learn
Matplotlib
Code Block language bash theme Midnight $ cat /usr/local/lcls/package/python/python2.7.13/linux-x86_64/src/site-packages/matplotlib-requirements.txt cycler functools32 matplotlib numpy pyparsing python-dateutil pytz six subprocess32
Top-level requirements*:
* - cx_Oracle >= 5.3.0 requires 64-bit Oracle 11.2 client or later. The proper ORACLE_HOME and related environment settings must be configured before
attempting to install the package via pip:Code Block language bash theme Midnight $ source /usr/local/lcls/tools/oracle/oracleSetup-R11.2.0.4.bash
Code Block language bash theme Midnight $ cat /usr/local/lcls/package/python/python2.7.13/linux-x86_64/src/site-packages/site-requirements.txt # package-specific requirements -r ./Flask-requirements.txt -r ./ipython-requirements.txt # global package requirements alembic Babel Cheetah Jinja2 Markdown MarkupSafe Pygments alabaster backports.ssl_match_hostname backports_abc certifi click cothread colorama cx_Oracle cycler decorator docutils dpkt first functools32 imagesize ipython ipython_genutils Mako mpmath nose numpy pandas pathlib2 pexpect pickleshare plex pydm pyepics == 3.2.7 pyqtgraph python-dateutil python-editor pytz PyYAML QtPy requests roman scipy scikit-learn simplegeneric singledispatch sip six snakefood snowballstemmer SQLAlchemy subprocess32 sympy tabulate tornado traitlets vcversioner virtualenv wsgiref -r ./jupyter-requirements.txt -r ./pip-tools-requirements.txt -r ./scikit-learn-requirements.txt -r ./matplotlib-requirements.txt
...