Managing Users on the Cluster

In order for someone to gain access to a cluster, s/he must first be given a user account. The cluster administrator can manage user accounts with the same tools that are available with most Linux distributions. User access to cluster resources can also be controlled by the cluster administrator.

This chapter discusses the tools and commands for managing user accounts and controlling access to cluster resources.

Managing User Accounts

Adding New Users

The useradd command enables you to add a new user to the system. This command takes a single argument, which is the new user’s login name:

[root@cluster ~] # useradd <username>

This command also creates a home directory named /home/<username>.

After you add the user, give them a default password using the passwd command so that they will be able to log in. This command takes a single argument, which is the username:

[root@cluster ~] # passwd <username>


It is good practice to give each user their own unique home directory.

Removing Users

To remove a user from your cluster, use the userdel command. This command takes a single argument, which is the username:

[root@cluster ~] # userdel <username>

By default, userdel does not remove the user’s home directory. To remove the home directory, include the -r option in the command:

[root@cluster ~] # userdel -r <username>


The userdel command will never remove any files that are not in the user’s home directory. To fully remove all of a user’s files, remove the user’s mail file from /var/spool/mail/, as well as any files the user may have in /tmp/, /var/tmp/, and any other directories to which the user had write permissions.

Managing User Groups

In addition to user accounts, you can also create user groups. Groups can be very powerful, as they allow you to assign resources to an arbitrary set of users. Groups are typically used for file permissions. However, you can also utilize groups to assign nodes to a specific set of users, thereby limiting which users have access to certain nodes. This section covers creating and modifying groups.

Creating a Group

Before you can add users to a group, you must first create the group. Groups can be created with the groupadd command. This command takes a single argument, which represents the name of the group:

[root@cluster ~] # groupadd <groupname>

Adding a User to a Group

Use the usermod command To add a user to a group. This command requires you to list all the groups the user should be a member of. To avoid accidentally removing any of the user’s groups, first use the groups command to get a list of the user’s current groups. The following example shows how to find the groups for a user named Smith:

[root@cluster ~] # groups smith
smith : smith src

After getting a list of the user’s current groups, you can then add them to new groups, for example:

[root@cluster ~] # usermod -G smith,src,<newgroup> smith

Removing a Group

To remove a group, run the groupdel command with the groupname as an argument:

[root@cluster ~] # groupdel <groupname>

Controlling Access to Cluster Resources

By default, anyone who can log into the master node of the cluster can send a job to any compute node. This is not always desirable. You can use node ownership and mode to restrict the use of each node to a certain user or group, including restricting compute node access to the master node.

What Node Ownership Means

Each node (including the master node) has user, group and mode bits assigned to it; these indicate who is allowed to run jobs on that node. The user and group bits can be set to any user ID or group ID on your system. In addition, the use of a node can be unrestricted by setting the user and group to “root”.

For the BProc unified process space, the node permissions “root” and “any” are equivalent. Node user access follows the normal Linux convention, i.e., the most restrictive access rule is the one used. Some examples:

  • user “root”, group “test”, mode 101 (u=1, g=0, o=1) — Users in the group “test” will not be able to access the node.

  • user “tester”, group “root”, and mode 011 (u=0, g=1, o=1) — The user “tester” will not be able to access the node.

  • user “tester”, group “test”, and mode 110 (u=0, g=1, o=1) — The user “tester” and users in the group “test” are the only non-root users able to access the node.


    In Linux systems, “other” is defined as anyone not listed in the user or group.

Checking Node Ownership

Display the current node access state by running the bpstat command:

[root@cluster ~] # bpstat -M
Node(s)  Status  Mode       User        Group
16-31    down    ---------- root        root
-1       up      ---x--x--x root        root
0-15     up      ---x--x--x root        root

The “User” column shows the user for each node and the “Group” column shows the group for each node. This display shows a cluster with default access permissions.

Setting Node Ownership

You can set node ownership with the bpctl command. Use the -S option to specify which node to change. Use either the -u option to change the user, -g option to change the group, or -m to change the mode. The only bit utilized for the mode is the execute bit. Following are some examples.

  • The following sets the user for node 5 to root:

    [root@cluster ~] # bpctl -S 5 -u root
  • The following sets all the compute nodes to be in the group beousers:

    [root@cluster ~] # bpctl -S all -g beousers
  • The following allows only the group beousers to access the compute nodes:

    [root@cluster ~] # bpctl -S all -m 010 -g beousers
  • The following disallows non-root users to execute on the master:

    [root@cluster ~] # bpctl -M -m 0110

For example:

[root@cluster ~] # bpctl -M -m 0110
[root@cluster ~] # bpctl -S 0-3 -g physics
[root@cluster ~] # bpstat -M
Node(s)  Status  Mode       User        Group
16-31    down    ---------- root        root
-1       up      ---x--x--- root        root
0-3      up      ---x--x--x root        physics
4-15     up      ---x--x--x root        root

See the Reference Guide for additional details on bpctl.

Using bpctl does not permanently change the node ownership settings. Whenever the master node reboots or systemctl restart clusterware reboots the cluster, the node ownership settings revert to the default of full, unrestricted access, or to the optional override settings specified by the nodeaccess directive(s) in the /etc/beowulf/config file. To make permanent changes to these settings, you must edit this file. For example, to make the above setting persistent, add the nodeaccess entries:

nodeaccess -M -m 0110
nodeaccess -S 0-3 -g physics

The Reference Guide and man beowulf-config provides details for the /etc/beowulf/config file.