Support for Diskful Nodes

Important

This software is a TECHNOLOGY PREVIEW that is being rolled out with limited features and limited support. Customers are encouraged to contact Penguin with suggestions for improvements, or ways that the tool could be adapted to work better in their environments.

The ClusterWare head nodes can be used to host packages that are needed during the installation of the compute nodes. In particular, the clusterware-node package must be installed if the node is to be fully integrated with ClusterWare. Other tools can also be staged on the head nodes for use on the compute nodes. For example, to integrate a compute node with the ClusterWare monitoring system, the telegraf and clusterware-telegraf packages are needed.

Each head node can offer a set of RPM, DEB, and TAR packages through an API end-point. Clients can then download those packages with a tool like curl.

RPM files can be automatically converted to TAR files for those systems that may not support other packaging systems. Note that while DEB packages may be provided by the head nodes, they cannot be converted to TAR files.

Note

The examples will use an IP address for the head node since, if ClusterWare is not yet installed, the node may not be able to resolve a head node's hostname.

Pre-Installer Script

In cases where the underlying system features are unknown, a "pre-installer" script can be downloaded from a head node. The script runs standard tests to determine the CPU-architecture, the base OS, and the type of packaging system available. The script provides the head node with the collected information and finally downloads and runs a customized installer that is better suited to that system.

To get a copy of the pre-installer for use in a custom script, download it through the API:

curl -o cw-preinst http://10.10.1.4/api/v1/install/client/installer

If the system configuration is already known, directly download and run a suitable installer script rather than starting with the pre-installer script.

Installer Scripts

After the pre-installer completes its tests, it posts the data to the head node and downloads the actual installer script. The pre-installer automatically runs this new script to complete the installation.

While the pre-installer will attempt to find the relevant system information, it may not be able to determine some features and will mark them as “unknown”. The head node uses the known features to determine the best installer. Even in the case where all features are properly detected, the head node may not have a fully customized installer for that particular combination of attributes. In those cases, it may reply with a more "generic" script that should work on a wide range of related or similar systems. For example, if the CPU-architecture is unknown but RPM support is detected, then the installer may leverage RPM-based installation of packages since it can assume that the RPM tooling will detect and install suitable packages.

While the installer does enable various Systemd services, it does not start those services. It is often easiest to reboot the system so that Systemd starts everything in the correct order, but you can also start the cw-status-updater service manually.

To download a copy of the installer to use as a starting point for a custom script, use the same API endpoint as before, but append the optional information to the URL:

curl -o cw-installer http://10.10.1.4/api/v1/install/client/installer?arch=x86_64&os=rhel&pkg=rpm

where:

  • arch is one of: x86_64 or aarch64

  • os is one of: rhel (includes RedHat, Rocky, Centos), debian, cumulus, or sonic (the latter two are network operating systems)

  • pkg is one of: rpm, deb, or tar

The TAR installer should work for many Linux systems and can be downloaded with:

curl -o cw-installer http://10.10.1.4/api/v1/install/client/installer?pkg=tar

Installation Logs

For both the pre-installer and installer scripts, logging output is produced as the script executes to enable error triage if necessary. Both log files are written to the /tmp directory, named as CLUSTERWARE_PRESINSTALL_LOG and CLUSTERWARE_INSTALL_LOG.

If the pre-install script is skipped (for example, if the TAR-based installer is directly selected through the API), then only the CLUSTERWARE_INSTALL_LOG is produced.

Head Node Preparation

As of 12.3.0, head nodes contain a "client packages" directory, /opt/scyld/clusterware/clientpkgs, with subdirectories for the installers as well as DEB, RPM, and TAR packages. By default, the package directories are empty, but admins can populate them in a variety of ways:

  • By copying files from the ClusterWare ISO (the ScyldPackages directory)

  • By running dnf commands to download relevant packages

  • By manually downloading RPMs from other sources

Any RPM files should be put in the rpm directory and DEB files in the deb directory. The tar directory can be left empty as the system will auto-convert RPMs to TAR files when requested by a node.

RPM and DEB Installations

The RPM and DEB installers install packages with the related tools: rpm or apt. This ensures that any dependencies are met, pre- or post-installation scripts are run, cross-package trigger scripts are run, etc.

To download the client packages inside a custom script, use the "download" endpoint along with the package type:

curl -o mypkg.rpm http://10.10.1.4/api/v1/install/client/download/rpm/mypkg

TAR Installations

To support a wider range of systems, ClusterWare provides an RPM-to-TAR conversion process for any RPMs in the head node's client-package repository. The TAR installer automatically downloads the related TAR file and unpacks it into /opt/scyld/<package-name>. Any files that should be installed in other directories are located in the ./pkg directory along with any pre- or post-installation scripts. The TAR installer automatically copies relevant files from ./pkg into /usr, /lib, and /etc as needed. It also automatically runs any post-install scripts.

The RPM-to-TAR conversion extracts any cross-package trigger scripts that were present in the original RPM, but the TAR installer does NOT automatically run those scripts. RPM refers to these as "triggerin" scripts and they often modify the installation process based on the presence of other packages on the system. Admins should check for any such trigger scripts and determine if they are needed for their particular system.

To download the client packages inside a custom script, use the "download" endpoint along with the TAR package type:

curl -o mypkg.tar.gz http://10.10.1.4/api/v1/install/client/download/tar/mypkg

The head node automatically converts RPM to TAR if it cannot find a pre-existing TAR package. Once converted, the TAR package is cached on the head node for use by other compute nodes.