bpctl – Control the operational state and ownership of compute nodes.
bpctl [-h, –help ] [-v, –version ] [-f] [-M, –master ] [-S num, –slave num ] [-s state, –state state ] [-m mode, –mode mode] [-u user, –user user] [-g group, –group group] [-H, –halt] [-P, –poweroff, –pwroff] [-R, –reboot ] [-O, –orphan ] [-C r2c-state, –completion r2c-state] [-I idle-threshold, –idle idle-threshold]
This utility is part of the
BProc package and is installed by
default. It allows the root user to modify the state of the compute
nodes. Compute nodes may be in one of eight states:
The states are described as follows:
No communication with compute node, and prior node state is unknown.
Node has initialized communication and started but not completed the node_up script. This state is not commandable. It is status information only.
Node is communicating and has completed the node_up script without errors.
Node is communicating and encountered an error while running the node_up script.
Node is communicating and the cluster administrator has marked the node as unavailable to non-root users.
Node will do a software reboot. Node status will show reboot through start of machine shutdown until node_up script has begun.
Node has been commanded to halt. This command causes the node CPUs to execute the halt machine instruction. Once halted the node must be reset by external means to resume normal operations.
- poweroff, pwroff
Node will power off. This command is valid for nodes that meet the ATX specification. This command requires BIOS support. Non-ATX machines may reboot on this command.
Normally the node will transition from
and will remain
up until commanded otherwise.
up is the
operational state for user programs. User
BProc commands will be
rejected if the node is not
BProc supports a simplified user and group compute node access
scheme. Before any action is taken on a node,
BProc checks if the
user or group match. If either is matched the user action is processed.
Note, normal file permissions are still in affect on each node.
BProc permissions simply allow users to execute a program on a node.
Root bypasses the check and always has access.
User and group changes made with bpctl remain in effect until the affected
node(s) are restarted. After a restart, the user and group information is read
/etc/beowulf/config file. For persistent changes, you must edit
the config file. Changes to the config file take effect when you issue a
reload or you reboot the nodes via a
reload, running jobs will not be affected unless they start a new
process and are denied node access based on the permission changes.
Whenever the ClusterWare daemons are restarted, all nodes are initialized
down state and node history is lost. When this occurs,
previously communicating nodes will reboot and attempt to re-establish
communication after the “ping timeout”, which by default is 30 seconds.
The following options are available to bpctl:
Print the command usage message and exit. If -h is the first option, all other options will be ignored. If -h is not the first option, the other options will be parsed up to the -h option, but no action will be taken.
Print the command version number and exit. If -v is the first option, all other options will be ignored. If -v is not the first option, the other options will be parsed up to the -v option, but no action will be taken.
Fast mode. Whenever possible, do not wait for acknowledgment from compute nodes.
Specifies that the remaining options apply to the master node.
- -S num
Specifies that the remaining options apply to the specified compute node. The num can range from 0 to the total number of nodes minus one.
- -s state
Set the node to the indicated state. Valid state values are
pwroff. Setting state to
downcauses the node to reboot due to a communications timeout after the “ping timeout” interval, which by default is 30 seconds.
- -m mode
Set the permission bits for the indicated node. Only the Execute mode bits are recognized, i.e., a logical or’ing of octal values 001, 010, and/or 100.
- -u user
Set the user id for the indicated node. Will reject invalid users. Numbers or strings may be used. A numeric user id will be converted to a name if the name is known.
- -g group
Set the group id for the indicated node. Will reject invalid groups. Numbers or strings may be used. A numeric group id will be converted to a names if the name is known.
- -H, --halt
Halt the indicated node.
- -P, --poweroff, --pwroff
Power off the indicated node.
- -R, --reboot
Reboot the indicated node.
- -O, --orphan
Direct the indicated node to become an immediate orphan.
- -C r2c-state, --completion r2c-state
Turn run-to-completion mode on or off for the nodes specified by
-Snum. Acceptable r2c-state values are
on(an “orphaned” node stays up indefinitely, until manually rebooted),
off(an “orphaned” node reboots immediately), or a positive number of seconds of “effectively idle” time that an orphaned node will wait until rebooting.
- -I idle-threshold, --idle idle-threshold
Override the default cpu usage percentage threshold that an “orphaned” compute node uses to determine whether or not the node is “effectively idle”.
When a compute node becomes an “orphan” and the r2c-state specifies that the node reboot after the specified number of “effectively idle” seconds, BProc periodically determines how much cpu usage has occurred during the preceding interval (which is nominally 10 seconds). If the cpu usage is above the idle-threshold percentage, then the time-until-reboot is reset back to r2c-state seconds. The idle-threshold value must be a positive numeric value, and it may be an integer or a floating-point number. A too-low value means BProc will mistakenly interpret trivial amounts of cpu usage (e.g., executed by daemons that wake up and check for work) as being significant, and thus the node may never reboot. A too-high value means BProc will mistakenly interpret significant cpu usage as being insignificant, and thus the node may reboot while a low-usage process is doing important work.
This command will cause all nodes to reboot:
[root@cluster ~] # bpctl -S all -s reboot
This command returns an error, because boot is not commandable:
[root@cluster ~] # bpctl -S 4 -s boot Non-commandable node state: boot
The following sets nodes 3 and 4 ownership to user “foo”, which must be a valid user:
[root@cluster ~] # bpctl -S 3-4 -u foo
The following sets permission on the master node to allow only user root to execute on the master node, e.g., to disallow a non-root user to execute on a compute node and bpsh a command to execute on the master:
[root@cluster ~] # bpctl -M -m 100
And this resets permission on the master node to allow any user to execute on the master node:
[root@cluster ~] # bpctl -M -m 111
This command resets the run-to-completion timeout to five minutes, and sets the “effectively idle” cpu usage percent to 1.5%:
[root@cluster ~] # bpctl -C 300 -I 1.5
Upon successful completion, bpctl returns 0. On failure, an error
message is printed to
stderr and bpctl returns 1.