Versions Compared

Old Version 59

changes.mady.by.user Valerio Mariani

Saved on

compared with

New Version 60

changes.mady.by.user Valerio Mariani

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  • Export the base lcls-i environment to a YAML file:

    Code Block
    conda env export > ana-4.0.44-py3.yaml

  • Run a script similar to the following to pin the main lcls-i packages from  at the right version:

    Code Block
    import yaml

with open("ana-4.0.44-py3.yaml", "r") as fh:
      curr_env = yaml.safe_load(fh)

dep_dictionary = {}
for entry in curr_env['dependencies']:
    items = entry.split("=")
    dep_dictionary[items[0]] = items[1]

with open("/cds/sw/ds/ana/conda1/manage/jenkins/ana-env-py3.yaml", "r") as fh:
    base_env = yaml.safe_load(fh)

new_dependencies = []
for entry in base_env['dependencies']:
    items = entry.split("=")
    new_dependencies.append(f"{items[0]}={dep_dictionary[items[0]]}")

base_env["dependencies"] = new_dependencies

with open("ana-4.0.44-py3-ami2.yaml", "w") as fh:
    yaml.dump(base_env, fh)
  • Add to the ana-4.0.44-py3-ami2.yaml file the following dependencies, at the version that the are in the current lcls-II version:
    • networkfox
    • asyncqt
    • amityping
    • mypy
    • setproctitle.
    • jedi
    • sympy
    • p4p
    • pyqode.python1
    • pint

    • pyfftw

      For example, in the case of 4.0.44, the following lines were added to the file (the version numbers come from the current lcls-ii development environment ps-4.5.16):

      Code Block
      - networkfox=0.0.8
- asyncqt=0.8.0
- amityping=1.1.11
- mypy=0.961
- setproctitle=1.2.3
- jedi=0.17.2
- sympy=1.10.1
- p4p=3.5.5
- pyqode.python=2.12.1
- pint=0.18
- pyfftw=0.13.0

  • Remove the version from the compiler dependency, so that it looks just like the following, without any version number:

    Code Block
    - compilers

  • Use the resulting yaml file to generate the environment, using the libmamba solver:

    Code Block
    conda env create --experimental-solver=libmamba -n ana-4.0.44-py3-ami2 --file ana-4.0.44-py3-ami2.yaml

Environments on S3DF

When creating an LCLS-II release on S3DF, the following lines

in the setup_env.sh file need to be specifically adapted to S3DF

Code Block
source /sdf/group/lcls/ds/ana/sw/conda2/inst/etc/profile.d/conda.sh
export CONDA_ENVS_DIRS=/sdf/group/lcls/ds/ana/sw/conda2/inst/envs
export DIR_PSDM=/sdf/group/lcls/ds/ana/

NOTE / TODO: DIR_PSDM should point to the directory containing the "detector" directory. We should make sure that the content of the detector directories on psana (/cds/group/psdm/detector) and sdf (/sdf/group/lcls/ana/detector) match. Currently, their content seem different: the size of the directory on psana is 1.3Tb on psana and 2.3TB on S3DF

Utility Scripts

The following two python scripts can be used to explore the content of a repodata.json file, which can be downloaded from a conda online package repository and lists in detail the content of the repository

...