You are viewing an old version of this page. View the current version.

Compare with Current View Page History

Version 1 Next »

There are two options to get MPI working at Nersc using Shifter. 

  1. Manual installation of MPICH followed by pip installation (with no binary option) of mpi4py
    This assumes that you install both MPICH and mpi4py in the conda environment that will be used to install psana. Choosing this way will NOT require you to uninstall MPICH (–force flag) at the end of docker build script.
  2. Use conda install (from conda-forge)
    conda create --name lcls2_base mpich=3.3.2 psana (or psana-conda for lcls1)
    Choosing this way, you'll need to uninstall mpich at the end of the docker script.

We are currently investigating why (1) works but (2) does not. 

The correct installation will show ldd of mpiXX.so as:

 

(lcls2_base) monarin@nid02517:~$ ldd /img/conda.local/miniconda3/envs/lcls2_base/lib/python3.7/site-packages/mpi4py/MPI.cpython-37m-x86_64-linux-gnu.so 

 linux-vdso.so.1 (0x00002aaaaaad5000)

 libmpi.so.12 => /opt/udiImage/modules/mpich/lib64/libmpi.so.12 (0x00002aaaaacd5000)

 libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00002aaaab297000)

 libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00002aaaab4b6000)

 libxpmem.so.0 => /opt/udiImage/modules/mpich/lib64/dep/libxpmem.so.0 (0x00002aaaab8a7000)

 librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00002aaaabaab000)

 libugni.so.0 => /opt/udiImage/modules/mpich/lib64/dep/libugni.so.0 (0x00002aaaabcb3000)

 libudreg.so.0 => /opt/udiImage/modules/mpich/lib64/dep/libudreg.so.0 (0x00002aaaabf39000)

 libpmi.so.0 => /opt/udiImage/modules/mpich/lib64/dep/libpmi.so.0 (0x00002aaaac144000)

 libgfortran.so.5 => /opt/udiImage/modules/mpich/lib64/dep/libgfortran.so.5 (0x00002aaaac390000)

 libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00002aaaac808000)

 libgcc_s.so.1 => /opt/udiImage/modules/mpich/lib64/dep/libgcc_s.so.1 (0x00002aaaacba6000)

 libquadmath.so.0 => /opt/udiImage/modules/mpich/lib64/dep/libquadmath.so.0 (0x00002aaaacdbf000)

 /lib64/ld-linux-x86-64.so.2 (0x00002aaaaaaab000)

 libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00002aaaad000000)

 libz.so.1 => /opt/udiImage/modules/mpich/lib64/dep/libz.so.1 (0x00002aaaad204000)


 

  • No labels