Page History

...

source /sdf/group/lcls/ds/ana/sw/conda1-v3/manage/bin/psconda.sh  # conda1-v3 is a latest version of conda 23.10.0
conda deactivate
conda activate conda_build2build

Generate .tar.gz file with source code

...

NOTE: (Oct. 10, 2023) we are renaming the .condarc to .condarc_dontuse files in home-directories and directories like /cds/sw/ds/ana/conda2-v2/inst/.condarc to not implicitly depend on those.  So channels must be specified explicitly in the conda commands.

Need to create a feedstock for every new package.  LCLS2 feedstock packages are in (for example) https://github.com/slac-lcls/epix-feedstock.

 Addition on Jan. 5, 2024 by cpo:  I found that when I was building cameralink-gateway (as user cpo) that it was trying to write into /cds/sw/ds/ana/conda2/inst/envs/conda_build/pkgs/python-3.9.18-h0755675_1_cpython.  So I restored my .condarc to get it to work.  I guess I could have tried to run as “psrel” instead.  Valerio writes: "I think this is another of the bug in conda. I think it is tied to the environment being group writable.  I don't think that having a condarc file is bad per se. I also have one with the paths pointing to non-home directories with more space. I think what we should avoid is having channels in the condarc, because when we run a conda build, conda also takes those channels into consideration. To have better control on what channels I am using, I removed all the channels from my condarc file and I specify the channels manually, but I still use the condarc for everything else!".

Need to create a feedstock for every new package.  LCLS2 feedstock packages are in (for example) https://github.com/slac-lcls/epix-feedstock.

For rogue packages only, need to create a .tar.gz since For rogue packages only, need to create a .tar.gz since their git repos are not public:

• Checkout the package (remove the ".git" subdirectory because it is large).
• Create .tar.gz file with "tar cvfz epix-quad-1.2.0.tar.gz epix-quad/"
• Copy the .tar.gz to the directory where it can be seen: "cp epix-quad-1.2.0.tar.gz /reg/g/psdm/web/swdoc/tutorials/"
• Compute the sha256sum with "sha256sum epix-quad-1.2.0.tar.gz"Put this sha256 in the epix-quad-feedstock/recipe/meta.yaml.gz"
• Put this sha256 in the epix-quad-feedstock/recipe/meta.yaml

NOTE: To generate a small .tar.gz file:

• use the following command to ignore the large git-lfs files when cloning:  "GIT_LFS_SKIP_SMUDGE=1 git clone git@github.com:slaclab/cameralink-gateway cameralink-gateway-8.2.2 --recursive"
• remove the ".git" subdirectory because it is large

The pinning is in conda_build_config.yaml (our packages are towards the end of the file, the conda-forge pinnings are earlier).  The conda forge pinnings are obtained from files like: https://github.com/conda-forge/conda-forge-pinning-feedstock/blob/main/recipe/conda_build_config.yaml.  Valerio wrote a small throwaway python script to take the packages from our environment and them to the conda_build_config.yaml.  This script also pins the low-level underlying packages that conda-forge does not explicitly pin and adds them to conda_build_config.yaml.  This conda_build_config.yaml cannot trivially be used by GitHub actions (because it will pull the latest version from conda-forge) so we are using this file and building locally on psbuild-rhel7 (with infinite time we could write a custom action).

...

After creating the package then upload, e.g. with "anaconda upload -u lcls-ii yaml-cpp-0.5.3-h1d15853_72.tar.bz2".  May need to do "anaconda login" (once per year?)upload, e.g. with "anaconda upload -u lcls-ii yaml-cpp-0.5.3-h1d15853_72.tar.bz2".  May need to do "anaconda login" (once per year?).

Rogue Package Recipe Creation

Since we have moved the rogue package creation to be done with pip it is necessary to explicitly list all the packages in setup.py similar to this: https://github.com/slac-lcls/epix-hr-m-320k-feedstock/blob/main/recipe/setup.patch.  This patch file gets run when the conda builds the recipe.  All the relevant packages to be included in this patch file can be found by looking for directories containing __init__.py.

Devel Env Creation

Export the current environment to yml (with includes pinnings).  Then we can do "small tweaks" to versions.  But big version changes will break the solver.  If we have a big change, we have to start from scratch: use /cds/sw/ds/ana/conda2/manage/env_create.yaml in the usual fashion (but use libmamba), e.g.:

...