Introduction to Tools

This section describes the commonly used arguments and subcommands used by the various Scyld ClusterWare tools. These tools can be used by the cluster administrator and are not intended for use by the ordinary user.

Certain arguments are shared among nearly all the scyld-*ctl tools, and instead of repeatedly describing these arguments, we will cover them here. Many of these arguments control the general operation of the tools, i.e. by printing help (--help or -h), selecting targets (--all or -a, --ids or -i), changing the verbosity or client configuration (--verbose or -v, --quiet or -q, --config or -c), allowing a user to override basic connection details (--base-url, --user or -u), or changing output formatting (--show-uids, --human, --json, --pretty or --no-pretty). Many of these arguments are self-explanatory, but others are described below:

--all and --ids

Tools that accept the --all (short name -a) and --ids (short name -i) arguments operate on corresponding database objects. For instance, scyld-nodectl is used for manipulating node objects in the database, and scyld-attribctl is used for manipulating attribute groups.

As one might expect, --all can be used to make an alteration to all of a given class of objects at once. For example, to remove a given attribute such as _boot_style from all attribute groups, e.g.:

scyld-attribctl --all clear _boot_style

Alternatively, an administrator can specify objects by name, or UID, or truncated UID (at least the first 5 characters of the UID are required to reduce the chance of accidental selection). Certain object types can also be selected based on some core fields, e.g. MAC, IP, or index for nodes. Further, nodes can be selected using the node query language, e.g.:

scyld-nodectl --ids n[0-5] --ids 08:00:27:F0:44:35 ls

For convenience, many tools can be executed without explicitly selecting any objects. Specifically, query tools such as list will default to --all if no selection arguments are used, and many other tools will operate on a single object if only one object of the expected type exists in the system.

--config

All client tools accept a --config argument which can be used to specify a client INI file. By default several locations are checked for configuration INI files with each able to override variables from the previous files. The client configuration search order is:

  • /etc/scyldcw/settings.ini

  • /etc/scyldcw/${TOOL}.ini

  • ~/.scyldcw/settings.ini

  • ~/.scyldcw/${TOOL}.ini

  • the --config specified path

  • command line arguments

These configuration files should be INI formatted, and the [ClusterWare] section can contain the following variables:

client.base_url  - http://localhost/api/v1
client.sslverify - True
client.authuser  - $USER
client.authpass  - None
client.format    - human
client.pretty    - False

The base_url specifies the URL that the tools should use to connect to the head node's REST API and defaults to connecting to the standard location (http://localhost/api/v1) on the local machine. If the base_url specifies an HTTPS URL, then a client can disable SSL verification, but this is strongly discouraged as it bypasses the protections provided by HTTPS against impersonation and man-in-the-middle attacks. The authuser and authpass can be included to simplify authentication to the service, but be aware that specifying the authpass here may not be secure, depending on your environment.

The format argument affects the output format of data returned by the tools. The default value of "human" causes the tools to output an indented format with various computed values augmented with human-readable summaries. The alternative value of "json" will output the results as JSON formatted text, and the pretty argument can be used to turn on indentation for that JSON output.

--base-url and --user

Since an administrator may want to periodically connect to different head nodes or as a different user, command line arguments are provided to override those configuration settings. For example, the entire string passed to the --base-url argument is treated as a URL and is passed to the underlying Python requests library.

Any string passed to the --user argument will be split at its first colon, and the remainder of the string will be treated as the user's password. Providing a password this way is convenient, especially during testing, but is generally discouraged as the password could then be visible in /proc while the tool is running. If no password is provided either through command line or client configuration, then one will be requested when needed.

--show-uids, --human, --json, --pretty/--no-pretty

These arguments are used to change the tool output format, much like the corresponding client configuration variables described above. The --human and --json arguments override the client.format variable, and --pretty and --no-pretty can be used to override the client.pretty variable.

By default, tool output will show an object's name when referring to a named object, and the UID (or shortened UID) only if no name is defined. Using the --show-uids argument forces the display of full UIDs in place of more human-readable options. This is uglier, but occasionally useful to be absolutely certain about what object is being referenced.

--csv, --table, --fields

For ease of reading and automated parsing, the scyld tools can also produce output as CSV or in a table. Use the --fields argument to select fields to display and select from --csv or --table to print in your preferred format:

$ scyld-nodectl --fields "mac,Assigned IP=ip,BootConfig=attributes._boot_config" \
                --table ls -l
Nodes |               mac |  Assigned IP |  BootConfig
------+-------------------+--------------+------------
   n0 | 08:00:27:f0:44:35 | 10.10.24.100 | DefaultBoot
   n1 | 08:00:27:a2:3f:c9 | 10.10.24.101 | DefaultBoot
   n2 | 08:00:27:e5:19:e5 | 10.10.24.102 | DefaultBoot

The above demonstrates how to both assign column names and select nested values such as individual attributes.

Common Subcommand Actions

In addition to the above arguments, some subcommand actions are common among the scyld-*ctl tools as well: list, create, clone, update, replace, delete. The precise details of what additional arguments these subcommand actions accept may differ between tools, but the generally supported arguments are discussed here.

list (ls)

List the requested object names, and optionally with --long or -l will display object details. The --raw option will display the actual JSON content as returned by the ClusterWare API call.

create (mk)

Create a new object using name-value pairs provided either on the command line or passed using the --content argument described below.

clone (cp)

Copy existing objects to new UIDs and names. Individual fields in the new objects can be overridden by name-value or a --content argument described below.

update (up)

Modify existing objects altering individual fields in name-value pairs or a --content argument described below.

replace (re)

Much like update, but completely replace the existing objects with new objects from fields defined in name-value pairs or a --content argument described below.

delete (rm)

Delete objects.

The then argument

Various tools (most commonly scyld-nodectl, although also scyld-adminctl, scyld-attribctl, scyld-bootctl, and scyld-imgctl) accept the then argument, which serves as a divider of a serial sequence of multiple subcommands for a single invocation of the scyld-* tool.

For example,

scyld-nodectl -i n0 reboot then waitfor up then exec uname -r

that initiates a reboot of node n0, waits for the node to return to an "up" state, and then executes uname -r on the node.

If any subcommand in the sequence fails, then the tool reports the error, skips any subsequent subcommands, and terminates.

The --content argument

The --content argument can be passed to several of the tools described earlier and is always paired with an argument to accept name-value pairs that can override content values. The --content argument can be followed by a JSON string or by a file containing JSON formatted data, INI formatted data, or a text file where each object is represented by rows of name-value pairs. If the argument to --content is a filename, it must be prefixed with an '@' symbol.

For example, an administrator could create a new boot configuration as follows:

scyld-bootctl create --content \
          '{"name": "TestBoot", "kernel": "@/boot/vmlinuz-3.10.0-957.1.3.el7.x86_64"}'``

Of course, a boot configuration also requires an initramfs:

cat > content.ini <<EOF
[BootConfig]
initramfs: @initramfs-3.10.0-957.1.3.el7.x86_64.img
EOF

scyld-bootctl -iTestBoot update --content @content.ini

Adding nodes to the database one at a time is tedious for large clusters, and the --content argument can streamline this process. Below are examples of three different files that could be passed via the --content argument to add nodes with explicit indices to the database:

JSON:

[
  { "mac": "00:11:22:33:44:55", "index": 1 },
  { "mac": "00:11:22:33:44:66", "index": 2 },
  { "mac": "00:11:22:33:44:77", "index": 3 },
]

INI:

[Node0]
mac: 00:11:22:33:44:55
index: 1

[Node1]
mac: 00:11:22:33:44:66
index: 2

[Node2]
mac: 00:11:22:33:44:77
index: 3

Text:

mac=00:11:22:33:44:55 index=1
mac=00:11:22:33:44:66 index=2
mac=00:11:22:33:44:77 index=3

Although providing multiple objects at once makes sense for the create subcommand, the clone, update, and replace subcommands require a list of fields to alter and will collapse multiple objects into one set of variables. For example:

[
  { "name": "TestBoot" },
  { "kernel": "@/boot/vmlinuz-3.10.0-957.1.3.el7.x86_64" },
  { "name": "AnotherBoot" }
]

when passed to scyld-bootctl would result in the selected boot configuration(s) being renamed to "AnotherBoot" and assigned the /boot/vmlinuz-3.10.0-957.1.3.el7.x86_64 kernel.

Files in database objects

In the ClusterWare database, boot configurations and images both contain references to files, either a kernel and an initramfs, or a root file system. The files themselves are not stored in the database but instead are referenced by the system on backend storage through plugins, such as the local_files plugin that works with locally mounted storage through the POSIX API.

When listing the details of a database object containing a file reference, the reference will be shown as a dictionary containing the file size, modification time, checksum, and an internal UID. To explore this we will start by listing a boot configuration created earlier in this document:

$ bin/scyld-bootctl ls -l
Boot Configurations
  TestBoot
    initramfs
      chksum: aa1161aa52b98287a3eac4677193c141a3648ebc
      mtime: 2019-02-09 21:13:08 UTC (0:00:52 ago)
      size: 20.0 MiB (20961579 bytes)
      uid: d247e4aa1fde4ac3853e78c0f7683947
    kernel
      chksum: 5a464d2a82839dac21c0fb7350d9cb0d055f8fed
      mtime: 2019-02-09 21:08:34 UTC (0:05:26 ago)
      size: 6.3 MiB (6639808 bytes)
      uid: fc96da5531b94038b38d7ef662b34947
    last_modified: 2019-02-09 21:08:34 UTC (0:05:26 ago)
    name: TestBoot
    release: 3.10.0-957.1.3.el7.x86_64
    uid: 4978077e53b944b38c4cda007b9b97b7

Files have been uploaded to both the initramfs and kernel fields. The chksum fields are the SHA-1 output and are used to detect data corruption, not as a security feature. The mtime is the UTC timestamp of the last time the underlying file was modified, and the size field is the size of the file in bytes. The uid field is how the object is referenced within the ClusterWare system and is the name passed to whatever plugin is interfacing with underlying storage. In the case of the local_files plugin, this is used as the name of the file on disk.

Because this output was generated for human readability, some fields (last_modified, mtime, size) have been augmented with human readable representations. Also, the release field was determined by examining the contents of the kernel file when it was uploaded.