bpctl

Name

bpctl – Control the operational state and ownership of compute nodes.

Synopsis

bpctl [-h, –help ] [-v, –version ] [-f] [-M, –master ] [-S num, –slave num ] [-s state, –state state ] [-m mode, –mode mode] [-u user, –user user] [-g group, –group group] [-H, –halt] [-P, –poweroff, –pwroff] [-R, –reboot ] [-O, –orphan ] [-C r2c-state, –completion r2c-state] [-I idle-threshold, –idle idle-threshold]

Description

This utility is part of the BProc package and is installed by default. It allows the root user to modify the state of the compute nodes. Compute nodes may be in one of eight states: down, boot, up, error, unavailable, reboot, halt, poweroff. The states are described as follows:

down

No communication with compute node, and prior node state is unknown.

boot

Node has initialized communication and started but not completed the node_up script. This state is not commandable. It is status information only.

up

Node is communicating and has completed the node_up script without errors.

error

Node is communicating and encountered an error while running the node_up script.

unavailable

Node is communicating and the cluster administrator has marked the node as unavailable to non-root users.

reboot

Node will do a software reboot. Node status will show reboot through start of machine shutdown until node_up script has begun.

halt

Node has been commanded to halt. This command causes the node CPUs to execute the halt machine instruction. Once halted the node must be reset by external means to resume normal operations.

poweroff, pwroff

Node will power off. This command is valid for nodes that meet the ATX specification. This command requires BIOS support. Non-ATX machines may reboot on this command.

Normally the node will transition from down to boot to up, and will remain up until commanded otherwise. up is the operational state for user programs. User BProc commands will be rejected if the node is not up.

BProc supports a simplified user and group compute node access scheme. Before any action is taken on a node, BProc checks if the user or group match. If either is matched the user action is processed. Note, normal file permissions are still in affect on each node. BProc permissions simply allow users to execute a program on a node. Root bypasses the check and always has access.

User and group changes made with bpctl remain in effect until the affected node(s) are restarted. After a restart, the user and group information is read from the /etc/beowulf/config file. For persistent changes, you must edit the config file. Changes to the config file take effect when you issue a ClusterWare service reload or you reboot the nodes via a restart. With reload, running jobs will not be affected unless they start a new process and are denied node access based on the permission changes.

Whenever the ClusterWare daemons are restarted, all nodes are initialized to the down state and node history is lost. When this occurs, previously communicating nodes will reboot and attempt to re-establish communication after the “ping timeout”, which by default is 30 seconds.

Options

The following options are available to bpctl:

-h

Print the command usage message and exit. If -h is the first option, all other options will be ignored. If -h is not the first option, the other options will be parsed up to the -h option, but no action will be taken.

-v

Print the command version number and exit. If -v is the first option, all other options will be ignored. If -v is not the first option, the other options will be parsed up to the -v option, but no action will be taken.

-f

Fast mode. Whenever possible, do not wait for acknowledgment from compute nodes.

-M

Specifies that the remaining options apply to the master node.

-S num

Specifies that the remaining options apply to the specified compute node. The num can range from 0 to the total number of nodes minus one.

-s state

Set the node to the indicated state. Valid state values are down, up, error, unavailable, reboot, halt, or pwroff. Setting state to down causes the node to reboot due to a communications timeout after the “ping timeout” interval, which by default is 30 seconds.

-m mode

Set the permission bits for the indicated node. Only the Execute mode bits are recognized, i.e., a logical or’ing of octal values 001, 010, and/or 100.

-u user

Set the user id for the indicated node. Will reject invalid users. Numbers or strings may be used. A numeric user id will be converted to a name if the name is known.

-g group

Set the group id for the indicated node. Will reject invalid groups. Numbers or strings may be used. A numeric group id will be converted to a names if the name is known.

-H, --halt

Halt the indicated node.

-P, --poweroff, --pwroff

Power off the indicated node.

-R, --reboot

Reboot the indicated node.

-O, --orphan

Direct the indicated node to become an immediate orphan.

-C r2c-state, --completion r2c-state

Turn run-to-completion mode on or off for the nodes specified by -S num. Acceptable r2c-state values are on (an “orphaned” node stays up indefinitely, until manually rebooted), off (an “orphaned” node reboots immediately), or a positive number of seconds of “effectively idle” time that an orphaned node will wait until rebooting.

-I idle-threshold, --idle idle-threshold

Override the default cpu usage percentage threshold that an “orphaned” compute node uses to determine whether or not the node is “effectively idle”.

When a compute node becomes an “orphan” and the r2c-state specifies that the node reboot after the specified number of “effectively idle” seconds, BProc periodically determines how much cpu usage has occurred during the preceding interval (which is nominally 10 seconds). If the cpu usage is above the idle-threshold percentage, then the time-until-reboot is reset back to r2c-state seconds. The idle-threshold value must be a positive numeric value, and it may be an integer or a floating-point number. A too-low value means BProc will mistakenly interpret trivial amounts of cpu usage (e.g., executed by daemons that wake up and check for work) as being significant, and thus the node may never reboot. A too-high value means BProc will mistakenly interpret significant cpu usage as being insignificant, and thus the node may reboot while a low-usage process is doing important work.

Examples

This command will cause all nodes to reboot:

[root@cluster ~] # bpctl -S all -s reboot

This command returns an error, because boot is not commandable:

[root@cluster ~] # bpctl -S 4 -s boot
 Non-commandable node state: boot

The following sets nodes 3 and 4 ownership to user “foo”, which must be a valid user:

[root@cluster ~] # bpctl -S 3-4 -u foo

The following sets permission on the master node to allow only user root to execute on the master node, e.g., to disallow a non-root user to execute on a compute node and bpsh a command to execute on the master:

[root@cluster ~] # bpctl -M -m 100

And this resets permission on the master node to allow any user to execute on the master node:

[root@cluster ~] # bpctl -M -m 111

This command resets the run-to-completion timeout to five minutes, and sets the “effectively idle” cpu usage percent to 1.5%:

[root@cluster ~] # bpctl -C 300 -I 1.5

Return Values

Upon successful completion, bpctl returns 0. On failure, an error message is printed to stderr and bpctl returns 1.