Bits & Bytes online Edition




Singularity: containers for HPC

Christian Guggenberger, Mykola Petrov

Singularity (http://singularity.lbl.gov/) is an open-source project which enables containers to be deployed in high-performance scientific computing environments. Through Singularity, scientists, developers and system administrators have the possibility to work with full control of their environments, including operating system, application software and dependencies. This environment can then be easily distributed and executed on other platforms in the form of containers.

The goals of Singularity are:

    • mobility of compute – the ability to create and maintain a workflow, within a container, which can later be executed on different hosts, Linux clusters and cloud services;
    • reproducibility – provide a unique representation of all the software and data files within a container, thus enabling the preservation and validation of data and scientific results;
    • user freedom – within a container a user can install applications and dependencies for workflows without impacting the host operation system;
    • security – Singularity blocks privilege escalation within and outside of the container;
    • support of HPC resources – Singularity natively supports InfiniBand, existing cluster filesystems (e.g. GPFS), and works seamlessly with resource managers (e.g. SLURM, SGE, etc.). This is possible because running a Singularity container is just like running any other command on the host HPC system.

Singularity utilizes container images, however, numerous other formats are also supported (docker images, tar.zg, cpio etc). The mechanisms to obtain the images include the standard URI format (i.e. http(s) download) as well as downloads from Docker and Singularity image registries.

In order to provide Singularity for researchers of the Max Planck Society, the MPCDF has deployed Singularity on several resources. The Singularity software is now available on the Draco HPC cluster as well as on several other systems. Please contact MPCDF helpdesk, if you want to use Singularity on Draco. More information, including example use cases and problems that Singularity can solve, can be found on the MPCDF software home page.