MPI

• We maintain our own installation of openmpi to make it lsf aware and build with verbs
• We add lsf_verbs to the package name
• It appears possible to build other mpi vendor implementations for lsf, google lsf aware mpi

 Updating MPI

• We will maintain the file manage/config/package_build_order
• This will tell us the order to rebuild packages, ie, openmpi, szip, hdf, mpi4py, h5py, pytables.
  • Usually szip doe not need to be rebuilt

Build MPI Package

• from the conda root dir, in a dev miniconda, do: ana-rel-manage -c bld-pkg -r manage/recipes/system/openmpi-1.10.3
• Do this on rhel5/6/7
• We'll build packages in the dev and install them in both dev and prod

Test Built MPI Package

• From a psana node with bsub,
  
  • sudo as psreldev
  • setup_condadev7
  • source activate _test
    • Gets you in the _test environment that was just used to build openmpi
  • cd $CONDA_ROOT/test/mpi
  • mpicc one of the C programs
  • bsub -n 3 mpirun -o output.log ./a.out
  • Check that it ran correctly