Appendix: Using SingularityΒΆ

Singularity is available in Scyld ClusterWare by installing the singularity-scyld RPM, which is built from source developed by Sylabs Inc., or by installing the singularity RPM found in the EPEL yum repository. See for their extensive documentation.

The following example creates a Singularity container openmpi.sif containing openmpi3.1, and placing that container in a bootable image.

First create the openmpi.def Singularity definition file, then use that file to create the container:

# Use quoted "EOF" for bash to avoid % and $ expansions; just EOF for sh.
cat <<-"EOF" >openmpi.def
BootStrap: yum
OSVersion: 7
Include: yum

    /etc/yum.repos.d/clusterware.repo /etc/yum.repos.d/clusterware.repo


    #   If instead using "OSVersion: 6" instead of "OSVersion 7" above,
    #   then for any subsequent `rpm` or `yum`, add:
    # rpm --rebuilddb
    echo "Installing openmpi3.1-gnu rpm"
    yum -y install openmpi3.1-gnu
    exit 0

# Create the Singularity chroot "/tmp/openmpi" in which updates can be made.
sudo singularity build --sandbox /tmp/openmpi openmpi.def

# Make a sample update: build an openmpi test program inside the chroot.
sudo singularity exec -w /tmp/openmpi \
     mpicc -o /usr/bin/ring /opt/scyld/openmpi/3.1.3/gnu/examples/ring_c.c

# Finalize the sandbox chroot into the Singularity container "openmpi.sif".
sudo singularity build openmpi.sif /tmp/openmpi

Create a bootable image that hosts the Singularity container and can execute openmpi applications:

# Clone a new image instead of modifying an existing image.
scyld-imgctl -i DefaultImage clone name=SingularityImage

# Install needed packages inside the new image.
scyld-modimg -i SingularityImage --freshen --overwrite --no-discard \
             --install singularity-scyld,openmpi3.1-gnu --upload

# Now get into the chroot of the Singularity image.
scyld-modimg -i SingularityImage --chroot --overwrite --upload --no-discard

# Inside the root, add your userid (e.g., "myuserid") if necessary, which
#   creates a /home/myuserid/ directory, and import the Singularity container file.
useradd myuserid
scp myuserid@localhost:/home/myuserid/openmpi.sif /home/myuserid/

Boot nodes n0 and n1 with SingularityImage:

scyld-bootctl -i DefaultBoot clone name=SingularityBoot
scyld-bootctl -i SingularityBoot update image=SingularityImage
scyld-nodectl -i n[0-1] set _boot_config=SingularityBoot
# Now reboot nodes n0 and n1
scyld-nodectl -i n[0-1] reboot

When the nodes are up, then initialize passphrase-less key-based access, as described in OpenMPI, MPICH, and/or MVAPICH.

Now you can run the ring program from n0 (or n1):

# logged into n0, or using a job scheduler
mpirun -np 2 --host n0,n1 singularity exec openmpi.sif /usr/bin/ring

Or from the head node:

# If not already installed
sudo yum install singularity-scyld openmpi3.1-gnu --enablerepo=scyld*

module load openmpi/gnu/3.1.3
mpirun -np 2 --host n0,n1 singularity exec openmpi.sif /usr/bin/ring