Detailed Installation Instructions

This chapter provides detailed instructions for installing Scyld ClusterWare. This software installation is intended for the first computer (“node”) of the cluster, which functions as the “master node” to control and monitor other nodes and distribute jobs.

Scyld ClusterWare is installed on the master node that is running with a base distribution of RHEL7 or CentOS7.

It is assumed that you are familiar with the concepts outlined in the previous chapters, and that you have correctly assembled the hardware for your cluster. If this is not the case, please refer to the previous chapters to acquaint yourself with Scyld ClusterWare, and then verify that your hardware configuration is set up properly.

Red Hat Installation Specifics

During a RHEL7 installation, you have the option to configure various aspects of the installation to support Scyld ClusterWare. Important points include the following:

• Disk partitioning — Scyld recommends letting the installer automatically partition the disk; refer to the Red Hat documentation if you plan to manually partition instead.

• Network interface configuration — To support your Scyld cluster, you need to configure one interface dedicated to the external public network (typically eth0) and one to your internal private cluster network (typically eth1). Detailed instructions are provided in the section on Network Interface Configuration.

• Network security settings — You can configure some of your firewall settings during a RHEL7 installation. Other settings needed to support a Scyld cluster must be made post-installation. Detailed instructions are provided in the sections on Network Security Settings and Trusted Devices.

• Package group selection — Scyld recommends installing all Red Hat packages. See Package Group Selection.

The following sections provide instructions and/or recommendations for specific portions of the RHEL7 installation that are relevant to an optimal Scyld ClusterWare installation. This guide does not cover all steps in the RHEL7 installation; you should refer to the Red Hat documentation for more complete information.

Network Interface Configuration

Tip

To begin, you must know which interface is connected to the public network and which is connected to the private network. Typically, the public interface is eth0 and the private interface is eth1.

A typical Scyld cluster has one interface dedicated to the external public network (typically eth0) and one dedicated to your internal private cluster network (typically eth1). It is important to properly to configure both of these interfaces to support your Scyld ClusterWare installation.

The network interface configuration screen will be presented to you during a RHEL7 installation. For an existing Red Hat installation, you can access the network configuration screens through the Red Hat Applications -> System Settings -> Network menu options.

Cluster Public Network Interface

DHCP is selected by default for all network devices, as shown below in the Red Hat Network Configuration Screen. For the public network interface (typically eth0), this option is recommended.

Figure 1. Public Network Interface (DHCP Default is Recommended)

However, if your external network is set up to use static IP addresses, then follow these steps to manually configure the interface:

1. In the Network Configuration screen, select the public network interface (typically eth0) in the Network Devices list, then click Edit to open the Edit Interface dialog box.

   

   Figure 2. Public Network Interface (Manual Configuration is Optional)

2. In the Edit Interface dialog box:

   a. Select the Activate on boot checkbox to initialize the specific network device at boot-time.

   b. Specify the IP address and netmask provided by your network administrator.

   When you have completed these settings, click OK to return to the Network Configuration screen.

3. In the Set the hostname area of the Network Configuration screen, select the manually radio button and provide a host name.

4. In the Miscellaneous Settings area of the screen, enter the gateway and primary DNS IP addresses provided by your Network Administrator.

Figure 3. Public Network Interface (Miscellaneous Settings for Manual Configuration)

Cluster Private Network Interface

Caution

For the private network interface (typically eth1), DHCP is shown as default, but this option cannot be used. The configuration tool Beonetconf requires a static IP address for the private interface. Therefore, you must configure the network interface manually and assign a static IP address and netmask.

The cluster will not run correctly unless the private network interface is trusted. You can set this interface as a “trusted device” when configuring the network security settings post-installation; see Trusted Devices.

1. In the Network Configuration screen, select the private network interface (typically eth1) in the Network Devices list, then click Edit to open the Edit Interface dialog box.

   

   Figure 4. Private Network Interface (Manual Configuration Required)

2. In the Edit Interface dialog box:

   a. Select the Activate on boot checkbox to initialize the specific network device at boot-time.

   b. Specify a static IP address. We recommend using a non-routable address (such as 192.168.x.x, 172.16.x.x to 172.30.x.x, or 10.x.x.x).

   c. If the public subnet is non-routable, then use a different non-routable range for the private subnet (e.g., if the public subnet is 192.168.x.x, then use 172.16.x.x to 172.30.x.x or 10.x.x.x for the private subnet).

   d. Once you have specified the IP address, set the subnet mask based on this address. The subnet mask must accommodate a range large enough to contain all of your compute nodes.

   When you have completed these settings, click OK to return to the Network Configuration screen.

3. In the Set the hostname area of the Network Configuration screen, you have the option to set the hostname automatically via the DHCP server or to provide one manually; this can be done according to your local standards.

The following figure illustrates a completed typical configuration for both the public and private network interfaces.

Figure 5. Public and Private Network Interfaces (Typical Configuration Completed)

Network Security Settings

Caution

The security features provided with this system do not guarantee a completely secure system.

The Firewall Configuration screen presented during the RHEL7 installation applies to the public network interface and should be set according to your local standards. This screen allows you to customize several aspects of the firewall that protects your cluster from possible network security violations.

The RHEL7 installer allows you to select some, but not all, of the security settings needed to support Scyld ClusterWare. The remaining security settings must be made post-installation; see Trusted Devices.

Figure 6. Security Settings During Installation

Scyld recommends setting a firewall for the public network interface (typically eth0). You can configure the following security settings during the Red Hat install:

Select from the following firewall options:

1. No Firewall — Allows all connections to your system and does no security checking. This option is not recommended unless you plan to configure your firewall after the installation.

2. Enable Firewall — Blocks any connections to your system that are not defaults or explicitly defined by you. By default, connections are allowed in response to outbound requests, such as DNS replies or DHCP requests.

Select services for which you want to allow possible connections. You can select any combination of the services listed.

Tip

If you plan to use SSH to connect to the master node, be sure that SSH is checked in the Trusted Services list.

Set the Enable SELinux? dropdown to “Disabled”.

If you chose to install a firewall, you must make the private network interface (typically eth1) a “trusted device” to enable all traffic on this interface to pass through the firewall. See Trusted Devices.

Package Group Selection

Caution

Scyld ClusterWare depends on certain Red Hat packages, and the Scyld installation may fail if the necessary Red Hat packages are not installed. Therefore, Scyld recommends that you install all Red Hat packages.

The Red Hat package selection screens enable you to select the particular software packages that you wish to install.

1. In the Package Installation Defaults screen, select the Customize… option.

   

   Figure 7. Customize Package Installation

2. In the Package Group Selection screen, scroll down to the Miscellaneous section. Select the Everything checkbox, then continue the installation process.

   

   Figure 8. Install Everything

   Tip

   To update an existing Red Hat installation to include all packages, insert the first Red Hat CD and invoke the Red Hat update program. Check the Everything box in the Package Group Selection screen, then continue with the update process.

Updating Red Hat or CentOs Installation

Update RHEL7 or CentOS7 either using yum, or using Red Hat or CentOS distribution media. Note that Penguin continually tests ClusterWare with new patches from Red Hat and CentOS. Visit the Penguin Computing Support Portal at https://www.penguincomputing.com/support to see the most recent errata fix tested with ClusterWare, and see any cautions about updated packages which may cause problems with ClusterWare.

Updating Using Yum

Use the following command:

[root@scyld ~]# yum update --disablerepo=cw*

(--disablerepo-cw* is used above in case the ClusterWare repo is already installed in /etc/yum.repos.d, you must exclude it during the yum update). You can also exclude other packages using the --exclude=$package parameter. See the yum man page for instructions on using yum. The CentOS web site also provides an online manual for yum at http://www.centos.org/docs/4/html/yum/.

Updating Using Media

If you update your system via distribution media, be sure to select an “upgrade install” rather than a “full install”, then follow the instructions provided with the media.

Tip

The just-installed newest base distribution kernel becomes the default in /etc/grub.conf. However, the Scyld ClusterWare includes a customized kernel that must be the kernel that is booted when running Scyld ClusterWare.

Scyld ClusterWare Installation

Scyld ClusterWare is installed using the Penguin Yum repository http://updates.penguincomputing.com/clusterware/. Each Scyld ClusterWare release is continuously tested with the latest patches from Red Hat and CentOS7. Before installing or updating your master node, be sure to visit the Support Portal to determine if any patches should be excluded due to incompatibility with ClusterWare. Such incompatibilities should be rare. Then, update RHEL7 or CentOS7 on your master node before proceeding (excluding incompatible packages if necessary) with installing or updating your Scyld ClusterWare.

Configure Yum To Support ClusterWare

The Yum repo configuration file for Scyld ClusterWare must be downloaded from the Penguin Computing Support Portal and properly configured:

Login to the Support Portal at https://www.penguincomputing.com/support.

Click on Download your Yum repo file to download this clusterware.repo file and place the it in the /etc/yum.repos.d/ directory.

Set the permissions:

[root@scyld ~]# chmod 644 /etc/yum.repos.d/clusterware.repo

With this setup complete, your master node is ready to retrieve Scyld ClusterWare installations and updates.

Install ClusterWare

You can use Yum to install ClusterWare and all updates up to and including the latest ClusterWare release, assuming you have updated your RHEL7 or CentOS7 base distribution as prescribed in the ClusterWare Release Notes.

1. Verify the version you are running with the following:

   [root@scyld ~]# cat /etc/redhat-release
   

   This should return a string similar to “Red Hat Enterprise Linux Server release 7.9” or “CentOS Linux release 7.9.2009 (Core)”.

2. Install the Scyld ClusterWare script that simplifies installing (and later updating) software, then execute that script:

   [root@scyld ~]# yum install install-scyld
   [root@scyld ~]# install-scyld
   

3. Configure the network for Scyld ClusterWare: edit /etc/beowulf/config to specify the cluster interface, the maximum number of compute nodes, and the beginning IP address of the first compute node. See the remainder of this guide and the Administrator’s Guide for details.

4. Reboot your system.

5. To verify that ClusterWare was installed successfully, do the following:

   [root@scyld ~]# uname -r
   

   The result should match the specific ClusterWare kernel version noted in the Release Notes.

Trusted Devices

If you chose to install a firewall, you must make the private network interface (typically eth1) a “trusted device” to enable all traffic on this interface to pass through the firewall; otherwise, the cluster will not run properly. This must be done post-installation.

After you have installed Red Hat and Scyld ClusterWare, reboot the system and log in as “root”.

Access the security settings through the Red Hat Applications -> System Settings -> Security Level menu options.

In the Security Level Configuration dialog box, make sure the private interface is checked in the “trusted devices” list, then click OK.

Tip

If you plan to use SSH to connect to the master node, be sure that SSH is checked in the “trusted services” list.

Figure 3. Security Settings Post-Installation

You are now ready to boot your compute nodes.

Enabling Access to External License Servers

1. Enable ipforward in the /etc/beowulf/config file. The line should read as follows:

   ipforward yes

2. Restart the cluster services as “root”:

::

[root@scyld ~]# systemctl restart clusterware

Post-Installation Configuration

Following a successful update or install of Scyld ClusterWare, you may need to make one or more configuration changes, depending upon the local requirements of your cluster. Larger cluster configurations have additional issues to consider. Accordingly, review the Release Notes sections titled Post-Installation Configuration Issues and Post-Installation Configuration Issues For Large Clusters for important detailed information.

Scyld ClusterWare Updates

You can use Yum update to update ClusterWare once you have upgraded your RHEL7 or CentOS7 base distribution. See Updating Red Hat or CentOs Installation or details on updating your base distribution, and Scyld ClusterWare Installation for how to set up the Yum repo configuration files.

To verify which distribution you are currently running, do the following:

[root@scyld ~]# cat /etc/redhat-release

Updating ClusterWare

1. It is advisable to update the base distribution prior to updating Scyld ClusterWare, taking care to exclude the base distribution’s kernel-* packages to avoid potentially updating to a newer kernel than is currently available in the Scyld ClusterWare yum repos:

   [root@scyld ~]# yum --disablerepo=cw* --exclude=kernel-* update
   

2. Update the Scyld ClusterWare package that contains a useful script that simplifies updating ClusterWare, then execute that script:

   [root@scyld ~]# yum update install-scyld
   [root@scyld ~]# install-scyld -u
   

3. Compare /etc/beowulf/config, which remains untouched by the Scyld ClusterWare update, with the new config.rpmnew (if that file exists), and examine the differences:

   [root@scyld ~]# cd /etc/beowulf
   [root@scyld ~]# diff config config.rpmnew
   

   and carefully merge the config.rpmnew differences into /etc/beowulf/config. Similarly, the preexisting /etc/beowulf/fstab may have been saved as fstab.rpmsave if it was locally modified. If so, merge those local changes back into /etc/beowulf/fstab

4. Reboot your system.

5. To verify that ClusterWare was installed successfully, do the following:

   [root@scyld ~]# uname -r
   

   The result should match the ClusterWare kernel version noted in the Release Notes.

6. Restart the compute nodes.