Installation & Administrator Guide

• Introduction
• Required and Recommended Components
• Installation and Upgrade of Scyld ClusterWare
  • Download the ClusterWare install script and related files
  • Execute the ClusterWare install script
  • Configure additional cluster administrators
  • Updating Base Distribution Software
  • Updating Scyld ClusterWare Software
    • Updating head nodes
    • Updating compute nodes
    • Updating ClusterWare 11 to ClusterWare 12
  • Updating Base Distribution Software
  • Updating Firmware
  • Services, Ports, Protocols
    • Apache
    • Chrony
    • DHCP
    • DNS
    • etcd
    • iSCSI
    • OpenSSH
    • Telegraf / InfluxDB
    • TFTP
• Backup and Restore
  • Backup and Restore of ClusterWare
  • Backup and Restore of the Database
• Node Images and Boot Configurations
  • Compute Node Fields
  • Compute Nodes IPMI Access
  • Boot Configurations
    • Deleting Boot Configurations
    • Freezing a Boot Configuration
  • PXEboot Images
    • Creating PXEboot Images
    • Recreating the Default Image
    • Modifying PXEboot Images
    • Caching in scyld-modimg
    • Updating the Kernel in an Image
    • Updating Drivers Inside Images
    • Capturing and Importing PXEboot Images
    • Deleting Unused Images
    • Freezing an Image
  • scyld-* Wrapper Scripts
    • Repos and Distros
    • Using Archived Releases
    • Using ISO Releases
    • Installing Software With Subscriptions
• Interacting with Compute Nodes
  • Node Creation with Known MAC address(es)
  • Node Creation with Unknown MAC address(es)
  • Changing IP addresses
  • Replacing Failed Nodes
  • Node Name Resolution
  • Executing Commands
  • Node Attributes
  • Node Names and Pools
  • Attribute Groups and Dynamic Groups
    • State Maps
• Using Kickstart
  • Create Local Repo
  • Create Boot Configuration
  • Kickstart Files
• Booting Diskful Compute Nodes
  • Installing the clusterware-node Package
  • Additional Support for Diskful Nodes
• Using RHCOS
• Common Additional Configuration
  • Configure Hostname
  • Managing Databases
    • Database Differences
  • Configure Authentication
  • Disable/Enable Chain Booting
  • scyld-nss Name Service Switch (NSS) Tool
  • Firewall Configuration
  • Configure IP Forwarding
  • Status and Health Monitoring
  • Install Additional Tools
• Securing the Cluster
  • Authentication
  • Role-Based Access Controls
  • Changing the Database Password
    • Changing the etcd Password
  • Compute Node Remote Access
  • Compute Node Host Keys
  • Encrypting Communications
    • Configure Encrypted Communication between Head and PXE Compute Nodes
    • Configure Encrypted Communication between Head and Diskful Compute Nodes
    • Configure Client Authentication between Head and Compute Nodes
  • Security-Enhanced Linux (SELinux)
    • SELinux On Compute Nodes
    • SELinux On Head Nodes
    • MLS Policy On Head Nodes
  • Security Technical Implementation Guides (STIG)
• Managing Multiple Head Nodes
  • Adding A Head Node
    • Join a non-ClusterWare server
    • Join a ClusterWare head node
    • After a Join
    • Cleaning up From Join Failures
  • Removing a Joined Head Node
    • Peer Downloads
  • Booting With Multiple Head Nodes
  • Copying boot configurations between head nodes
• Additional Software
  • Additional ClusterWare Software
  • Adding 3rd-party Software
  • Job Schedulers
    • Slurm
    • PBS TORQUE
    • OpenPBS
  • Kubernetes
  • OpenMPI, MPICH, and/or MVAPICH
• Troubleshooting ClusterWare
  • ClusterWare Log Files
  • Command-Line Monitoring of Nodes
  • Failing PXE Network Boot
  • Kickstart Failing
  • Head Node Filesystem Is 100% Full
    • Verify excessive storage is related to ClusterWare
    • Remove unnecessary objects from the ClusterWare database
    • Investigate InfluxDB retention of Telegraf data
    • Remove unnecessary PXEboot images, repos
    • Otherwise
  • Exceeding System Limit of Network Connections
  • etcd Database Exceeds Size Limit
  • Failing To Boot From Local Storage
  • IP Forwarding
  • Soft Power Control Failures
  • Head Nodes Disagree About Compute Node State
  • Applications Report Excessive Interruptions and Jitter
  • Managing Node Failures
    • Head Node Failure
  • Managing Large Clusters
    • Improve scaling of node booting
  • Finding Further Information
  • Contacting Penguin Computing Support