MPI
- We maintain our own installation of openmpi to make it lsf aware and build with verbs
- We add lsf_verbs to the package name
- It appears possible to build other mpi vendor implementations for lsf, google lsf aware mpi
Updating MPI
- We will maintain the file manage/config/package_build_order
- This will tell us the order to rebuild packages, ie, openmpi, szip, hdf, mpi4py, h5py, pytables.
- Usually szip doe not need to be rebuilt
Build MPI Package
- from the conda root dir, in a dev miniconda, do: ana-rel-manage -c bld-pkg -r manage/recipes/system/openmpi-1.10.3
- Do this on rhel5/6/7
- We'll build packages in the dev and install them in both dev and prod
Test Built MPI Package
- From a psana node with bsub,
- sudo as psreldev
- setup_condadev7
- source activate _test
- Gets you in the _test environment that was just used to build openmpi
- cd $CONDA_ROOT/test/mpi
- mpicc one of the C programs
- bsub -n 3 mpirun -o output.log ./a.out
- Check that it ran correctly
Overview
Content Tools