Page History

...

• We will maintain the file manage/config/package_build_order
• This will tell us the order to rebuild packages, ie, openmpi, szip, hdf, mpi4py, h5py, pytables.
  • Usually szip doe not need to be rebuilt

Build MPI Production Package

• from the conda root dir, in a dev miniconda, do: ana-rel-manage -c bld-pkg -r manage/recipes/system/openmpi-1.10.3-prod
• Do this on rhel5/6/7
• We'll build packages in the dev and install them in both dev and prod

...

Build MPI Debug Package

• I have build valgrind 3.11.0 by hand, it is in /reg/g/psdm/sw/conda/inst/external
• For each rhel[567], need to set the environment variable REDHATVER=5 or 6 or 7 before running the build,
• export these variables from psel706, psdev105 and psdev106 (7,6 and 5 respectively)
• from the conda root dir, in a dev miniconda, do: ana-rel-manage -c bld-pkg -r manage/recipes/system/openmpi-1.10.3-debug

Test New Production and Debug MPI Package

Test as psreldev

• From a psana node with bsub, as psreldev,
  
  • sudo as psreldev
  • setup_condadev7
  • source activate _test
    • Gets you in the _test environment that was just used to build openmpi
  • cd $CONDA_ROOT/test/mpi
  • mpicc one of the C programs
  • bsub -n 3 mpirun -o output.log ./a.out
  • Check that it ran correctly

Test environment as a user

• As prseldev, in say the dev7 conda install, make test environments for the production and debug mpi packages. If we have several builds of openmpi around, we need to specify the complete build string:
  
  • conda create -n testana python=2.7 openmpi=1.10.3=lsf_verbs_100
  • conda create -n testanadbg python=2.7 openmpi=1.10.3=lsf_verbs_dbg_100
• As a user, activate each environment
  • currently, as davidsch, I have a script setup_conda that takes an argument like dev7 to get into a specifiy conda install
  • on pslogin, checkout from git: git clone https://github.com/davidslac/test-lcls-conda-build-system.git
  • source activate testana
    • on a psana node, cd mpi
    • mpicc communicator.c
    • bsub -n 20 -I mpirun a.out
  • source activate testanadbg

    • mpiccvt communicator.c

      • (don't have to use vt, not sure if this is a good idea: users manual: openmpi_vampirtrace_usermanual.pdf)

    • test with valgrind:

      • bsub -n 20 -q psanaq -I mpirun valgrind a.out

      • more info: https://www.open-mpi.org/faq/?category=debugging#memchecker_what