Page History
- can't create non-root user in docker image because singularity/shifter map the root user to the true username. creates issues on laptops where mpi doesn't like to be run as root
- with both Shifter and Singularity you can do
mpirun -n 3 <container>to run on multiple nodes but not with Docker - psana1 containers are big (8GB for python2-10GB for python3)
- mounting right directories in the container is non-trivial for non-experts when using Docker. singularity Singularity helps with this by mounting the user's home dir and a set of dirs defined by the administrators.
- the following features are non-trivial for non experts when using Docker: graphical applications, hardware acceleration and CUDA. Singularity helps with all these
- singularity requires vagrant on macos
- some installations of MPI on the host define a variable called OPAL_PREFIX. In some cases this interferes with MPI inside the container, and needs to be unset before launching a container with MPI: unset
OPAL_PREFIX && mpirun -n 3 <container>(note by valerio: this seems fixed now) - precise version of cuda in the container must match the host
- major version of mpi in the container must match the host
- containers at NERSC are patched with NERSC-specific versions, we believe because of the interconnect, although this happens automatically when doing "docker pull" at NERSC
- we think for singularity each node has to access the container file on startup. we believe shifter has the ability to broadcast the container more efficiently to all nodes (presumably through a tree-like structure) to improve startup times. cpo's recollection is that the container-broadcast only worked for one of "mpirun shifter" or "shifter mpirun" (don't remember which).
Overview
Content Tools