...
- Virtual machines running in a cloud environment or on a farm made of virtual nodes
- just a simple VM for developers or end users
Containers
- a few technologies to consider Docker, Shifter (Nersk for HPC) and Singularity
- advantages over a full VM usually are:
- light weight
- better performance
- container has to be carefully built
- Docker: the container
- Shifter: NERSC container optimized for HPC
- Singularity: containers for scientist, compatible with Docker
- Another very interesting link : http://geekyap.blogspot.fr/2016/11/docker-vs-singularity-vs-shifter-in-hpc.html
Tabular Comparison
| Name | Docker | Singularity | Shifter |
|---|---|---|---|
| Main Goal | MicroServices, Enterprise applications | Application portability (one image with all dependencies) | Run Docker containers in HPC environment, Improve Docker security |
| UGE compatible |
but CC won't use it |  | |
| LSF compatible | | | |
| Security | User running docker commands needs to be in special docker group to gain elevated system access LSF improves the Docker security | User runs Singularity image without special privileges. | User run shifter image without special privileges. |
What systems need what kinds of containers?
...
- Notes for the SLAC farm
- Last purchase went into dev cluster
- many nodes @RHEL6, upgrade to RHEL7 and doing docker with this
- Still figuring out NFS/AFS sorted out with RHEL7. GPFS?
- It's good to come up with a plan because of security implications if NFS underneath.
- Use right docker (UID issues w/security)
- SLAC has a few nodes for testing docker.
- AFS on RHEL6 docker
- read files if world readable.
- NFS is hardest.
- Timeline for RHEL7, 12mo? 2018? (Matt)
- RHEL7 support is dodgy.
- Configuration stuff is hard part
- Last purchase went into dev cluster
- Notes for CC-IN2P3
- Now full support for Docker at Lyon (Fred)
Joris : Lyon wants to use Singularity because they have security issues with UGE + Docker.
Johan : the CentOS7 queue has Singularity available, will run some tests (for CTA...) soon - July 17th 2017.
...
Questions
- Joris : Is there some security issues with LSF & Docker (https://developer.ibm.com/storage/2017/01/09/running-ibm-spectrum-lsf-jobs-in-docker-containers/ )
- Joris : We need to verify the compatibility between Singularity ( Lyon CC ) and Docker
- etc.
...