Page History

...

1. GitHub Access Token

   
   A new set of python scripts has been developed to automate the creation of LCLS-II conda environment. They can be found here:
   

   /cds/sw/ds/ana/conda2/manage/buildenv

   The scripts need a valid personal GitHub token to be used (unfortunately, GitHub caps the number of API requests that can be done without a token. The token can be created on GitHub, after logging in:

   https://github.com/settings/tokens

   The token must be exported and available as an environment variable called GITHUB_ACCESS_TOKEN

   For an example, please see ~valmar's bashrc file
   
   

2. Package Version File

   
   In order to generate packages, a YAML file listing the reuired version of each package must be created. For a file with the package versions in the latest environments, see:
   /cds/sw/ds/ana/conda2/manage/buildenv/table.yaml

   A few lines as an example:

   ami: 2.4.7
   amityping: 1.1.7
   cameralink-gateway: 7.6.2
   epix: 0.0.3
   lcls2-pgp-pcie-apps: 2.2.0
   lcls2_timetool: 3.3.0
   epix-hr-single-10k: 3.1.1
   lcls2-epix-hr-pcie: 1.2.0
   ....

3. Preparing source tarballs

   
   Before building the feedstocks, source tarballs must be created for the rogue-related packages (which are private repos, so need source .tgz files generated) by running the prepare_source.py script.
   
   The script must be run using the conda_build environment (conda activate conda_build) It must also be run as a normal user because psrel cannot write to the /reg/g/psdm/web/swdoc/tutorials/.
   The script takes the package version file as an argument:
   

   python /cds/sw/ds/ana/conda2/manage/buildenv/prepare_source.py --package-version-file=/cds/sw/ds/ana/conda2/manage/buildenv/table.yaml

   
   The source tarballs are automatically generated and copied to /reg/g/psdm/web/swdoc/tutorials/
   
   PS: The script will clone the required repostiories in the current working directory!!!! Working in a temporary directory that can be later deleted is strongly advised
   
   
4. Generating packages
   
   The packages can now be built using the build environment script, again in the conda_build environment (conda activate conda_build):
   

   python /cds/sw/ds/ana/conda2/manage/buildenv/build_environment.py --generate-packages --package-version-file=/cds/sw/ds/ana/conda2/manage/buildenv/table.yaml

   
   The script will build the packages wave-by wave. For each wave, the script will clone the feedstocks one by one, make the necessary changes to the recipes automatically, run "conda smithy" and push the changes to the git repositories, triggering the building of the packages.  The individual builds can be seen at URLs like https://github.com/slac-lcls/libnl3-feedstock/actions
   
   The script will then check with GitHub every thirty seconds, and report the status of the build process ("Not started yet", "running", "success" or "failed"). It will waint until all builds have finished and are either in a "success" or "failed" state. If no build has failed, the script will then proceed to the next package "wave". Otherwise it will exit