Appendix: Managing Zero-Touch Provisioning (ZTP)

Important

Currently only supported for Cumulus switches.

Scyld ClusterWare supports ZTP (Zero-Touch Provisioning) of ONIE and related switches. Note that ZTP by itself does not provide a full, end-to-end control plane for cluster networking, but it is the first step in that direction, allowing for server-provided scripts to alter the configuration of connected switches.

Since the ZTP-capable switches are essentially Linux management systems attached to the switches, ClusterWare treats them as another node in the cluster. You can add them to the cluster using scyld-nodectl create and specifying the switch's MAC address. For example:

scyld-nodectl create mac=aa:bb:cc:00:11:22

which simplistically creates a new (switch) node in the default naming-pool and default group. This may not be the desirable approach, since it assigns a generic name like "n12" which is superficially indistinguishable from compute nodes "n0" through "n11". A better approach is to utilize the ClusterWare naming-pool and attribute-group functionality to assign a more self-identifying name and permit more efficient management of this and other ZTP-capable switches:

scyld-clusterctl pools create name=ztpswitch pattern=”switch{}”
scyld-nodectl create mac=aa:bb:cc:00:11:22 naming_pool=ztpswitch

which creates a new naming pool "ztpswitch" and configures the new node inside that pool with the name "switch0". Subsequent ZTP-capable switches can use the same naming-pool, which names them "switch1", "switch2", etc.

The cluster administrator can then use:

scyld-nodectl -i switch2 <action>
scyld-nodectl -i switch* <action2>

to perform an action on a specific switch or a common action on all switches in that naming-pool.

Configure each ZTP node to boot using a ZTP boot script. A boot script may be written in Bash or Python. As with other scripts, the first line should be #!/path/to/interpreter, e.g. #!/bin/bash. Some switches also allow Perl, Ruby, or a vendor-specific language. These scripts execute as user root on the switch and can execute commands supported by the switch, including triggering Puppet or Ansible runs, downloading files via wget or curl and manipulating or moving them on the switch, and more. After a successful execution, the script must return status 0.

ZTP boot scripts reside in /opt/scyld/clusterware/kickstarts/. Configure the boot script ztp_config.sh for the node switch0 using the specific prefix "ztp:":

scyld-nodectl -i switch0 set _boot_config="ztp:ztp_config.sh"

Since switch nodes are ClusterWare nodes, you can use attribute groups to configure this as well:

scyld-attribctl create name=ZtpSwitches
scyld-attribctl -i ZtpSwitches set _boot_config="ztp:ztp_config.sh"
scyld-nodectl -i switch0 join ZtpSwitches

which creates an attribute group "ZtpSwitches" and joins "switch0" into it. All members of that attribute group will boot the same ztp_config.sh script.

In a multi-headnode cluster, every head node should have the same ZTP boot script installed. Currently this must be done manually.

At boot time the ZTP-enabled node switch0 executes a DHCP query. The server sees the query, identifies the node using the client's MAC address in the DHCP request, recognizes the client as a ZTP-enabled node and the node's _boot_config's "ztp:", then builds a DHCP response that includes a URL of the form http://<SERVER_IPADDR>/boot/ztp_config.sh. The switch then uses standard web protocols to read the URL to download the script and execute it.

Per the Cumulus Linux guidelines, the script must include the phrase "CUMULUS-AUTOPROVISIONING", usually in a comment, in order to execute at ZTP boot. Other switch or NOS vendors may require similar keywords.

While the system may provide some limited logging that the ZTP script was run, it may make sense to log any/all command outputs to a known file for easier debugging and triage. A line such as exec >> /var/log/autoprovision 2>&1 in a bash script writes output to that log file for subsequent commands in the script.

Once a ZTP-switch has been successfully configured and the script returns status 0, it will not execute the ZTP boot script again, not even at the next reboot of the switch node. To force the switch to re-execute the boot script on the next reboot, ssh to the switch and execute sudo ztp -–reset.