/etc/beowulf/fstab – ClusterWare compute node filesystem control table
/etc/beowulf/fstab file on the master node contains a list of
filesystems to be mounted on compute nodes at boot time. Its purpose,
format, and contents are similar to the traditional
a few additional cluster-specific features.
fstab system is designed to keep all configuration
information on a master node. The
/etc/beowulf/fstab file is the
default for all compute nodes. Any optional node-specific
/etc/beowulf/fstab.N file overrides this default file for node
The root filesystem on each compute node is a
tmpfs filesystem that
is automatically sized for the available RAM.
In earlier versions of Scyld ClusterWare,
this root filesystem was explicitly declared in
fstab, but this is
no longer done.
The compute node’s root filesystem is used to dynamically cache binaries and libraries from the master, to provide space for /tmp and /var/tmp, to provide mountpoints for NFS mounts, etc. Although ClusterWare does not require a harddrive on a Scyld compute node, some clusters employ harddrive(s) for node-local persistent storage, for “scratch” storage to avoid having /tmp and /var/tmp consume tmpfs RAM, or for swap space to expand the available virtual memory space and thus reduce Out-of-Memory conditions.
fstab interacts with the mkfs and fsck directives in
/etc/beowulf/config file (see
man beowulf-config) to control
automatic creation or boot-time checking (and potentially repairing) of
compute node filesystems on node-local harddrives.
A directive mkfs always specifies to rebuild at boot time every
harddrive partition specified in
/etc/beowulf/fstab, and thus should
be used with great care so as to not automatically rebuild a partition
and thus destroy data that is expected to survive across compute node
reboots. Normally the default directive mkfs never is used.
A directive fsck full specifies to check and potentially repair at boot time every harddrive partition. Alternatively, fsck safe specifies to perform an fsck, but to not attempt any repairs; after boot, the cluster administrator may manually perform repairs as needed. A directive fsck never is the default, which specifies that no checking be done at boot time.
The syntax and layout is identical to the master node’s
file. The file contents are processed line by line. All blank lines and
lines that begin with a “#” are ignored. All other lines should have six
fields, separated by tabs or spaces.
The first field is the device to mount. For filesystems on local
harddrives, this should point to a
/dev entry, such as
/dev/hda2. If mounting an NFS filesystem, the device should be
specified as hostname:directory, where hostname is the IP address of the
NFS server, and directory is the path on the NFS server you want to
mount. If the NFS server is the master node, you can use “$MASTER” as
the hostname. Currently, hostname cannot be an actual alphanumeric host
/etc/beowulf/fstab is evaluated at boot time before the
compute node’s name service is initialized. For some special
filesystems, such as
devpts, the hostname can be set to
The second field is the mount point. For a swap partition, this should
be “swap”, but for all other filesystems, this must be a path that
begins with “/”. Any paths that you specify as mount points will be
automatically created by the
node_up script before it tries to mount
the filesystem. Ensure that you do not specify the same mount point on
more than one line, because this can cause problems. You can have
multiple lines that use “swap” as the mount point, but that is the only
exception to the rule.
The third field is the filesystem type. This should be “swap” for swap
partitions, or a standard Linux filesystem type (e.g., “ext2”, “ext3”,
“xfs”), or “nfs” for an NFS file system, or particular pseudo filesystem
types (e.g., “proc” for the
proc filesystem, “devpts” for the
devpts filesystem). Any filesystem that can normally be used by
Linux can also be specified here, but you must also take steps to create
the harddrive filesystems on the compute nodes before attempting to
The fourth field lists the mount options for the filesystem. All options should be comma-separated with no spaces. If you do not know of any specific options to use, then you should use the “defaults” keyword.
In addition to the mount options normally supported by Linux, one
additional option is supported by ClusterWare: “nonfatal”. Normally, any mount
failure results in an immediate abort of the node boot, and the node
state transitions from “boot” to “error”. Adding “nonfatal” to the
options overrides this behavior and allows the node boot to continue,
potentially to a node “up” state. However, because filesystem mounts
have in fact failed, the node may not actually have full functionality.
When using the “nonfatal” option, the cluster administrator is
encouraged to view the ClusterWare boot log files found in directory
/var/log/beowulf/ to discover potential mount failures and other
warnings or soft error conditions. The “nonfatal” option is useful for
harddrive filesystems when not all compute nodes share the same number
and partitioning of drives, or when NFS mounts might fail because an NFS
server is temporarily unavailable or the specified filesystem is not
The fifth and sixth fields are left there for compatibility with the
fstab format. These fields are not used at the moment, but
are required to be there. We recommend they both be set to “0”.
# This file is the fstab for nodes. # One difference is that we allow for shell variable expansions... # # Variables that will get substituted: # MASTER = IP address of the master node. (good for doing NFS mounts) # This is the default setup from beofdisk, once you setup your disks. #/dev/hda2 swap swap defaults,nonfatal 0 0 #/dev/hda3 / ext2 defaults,nonfatal 0 0 # These should always be added none /proc proc defaults 0 0 none /dev/pts devpts gid=5,mode=620 0 0 # NFS (for example and default friendliness) $MASTER:/home /home nfs nolock,nonfatal 0 0