root for conda work: /reg/g/psdm/sw/conda

Right now, can't build things once that work on all OS's, so managing rhel[567], also, since you can do things that screw up conda itself, I want a devel and production installation of the two that are separate, makes for 6 miniconda installs (sad)

Key subdirectories here:

  • build - the condarc's will point to subdirs here for the minicondas: dev-rhel5  dev-rhel6  dev-rhel7  prod-rhel5  prod-rhel6  prod-rhel7
  • channels - the channels
    • external-rhel[567] things like szip that we can't find in conda channels
    • system-rhel[567] openmpi that we have to build locally against lsf and to use verbs/infiniband
    • psana-rhel[567] ndarray, psalg, pdsdata, psana, etc
    • there will be softlinks here like psana -> psana-rhel5 to suggest that our rhel5 builds will be what we export publicly, if we can get that one channel to work for all linux?
  • develop - this has subdirs like
    • github/davidslac/conda - where I modified conda-env to work around an issue
    • rhel7/valgrind-3.11.0 - build a package
    • hdf5-1.8.17 - build a package, should be in a rhel[567] dir
  • downloads - 
  • envs - at one point I was going to have the miniconda installs in put there environments here, would help isolate packages, but should remove: JIRA PSRT-78
    • dev-rhel[567] and prod-rhel[567], the condarc's will point the installs here
  • inst  - this is where the miniconda installs will be, and external packages we can't make work in conda. subdirs
    • miniconda2-dev-rhel[567] and miniconda2-prod-rhel[567]
    • external
      • For example, valgrind is not relocatable. So right now, there is a external/valgrind/valgrind-3.11.0/rhel[567] set of directories
  • logs - from all the builds. Filenames like conda_build_external_szip-2.1-0_rhel5_linux-64_00.log
    record channel, package, and OS. By default repeated builds increment the _00 at the end.
    • conda_build_* most logs start with conda_build_channel,
    • external-build-pkg* to distinguish logs for building things in valgrind
    • develop-* logs for packages in the develop area
  • manage - software to manage, this is checked out in github. There is a readme with more recent info
    • config - template for condarc, tag files, etc?
    • pinned - soft links to the 'pinned' files that are in directories like envs/ana/conda-meta
    • recipes - recipes for packages we build locally
      • external - like our old external packages, but the ones that we'd rather not build ourselves, just doing it because we couldn't find a conda channel, or we wanted more control over the package build
        • szip-2.1 
        • guppy
      • psana
        • psalg
        • pdsdata
        • ndarray
        • psana - when we get around to it
      • system - anything that has system dependencies, like mpi, and then anything that depends on it, maybe we should roll this in with external?
        • openmpi
        • hdf5
        • h5py
        • mpi4py, etc
    • scripts the code that we run to help manage, in particular, the file
      • ana-rel-manage - just give it -h to see what to do
  • sandbox - scratch stuff to test things out, like figuring out rpath, etc
  • test - after activating a conda environment, do python run_tests.py from here.

  • No labels