Provisioning an RKE2 (Rancher Kubernetes Engine 2) cluster on vSphere

#rancher #kubernetes #rke #vsphere

In this article i will walk you down with steps to create RKE2 cluster on vSphere vCenter from Rancher UI.

Pre-requisites

Rancher nodes needs to communicate with vSphere vCenter on port 443.
Rancher nodes needs to communicate with RKE2 cluster nodes on port 22.

1. Installing packages in the template VM.

Create a new Ubuntu VM, perform below steps and later convert it into a template. We will let rancher use this template to create VM's.

Ensure below packages are installed in the template,
• curl
• wget
• git
• net-tools
• unzip
• apparmor-parser
• ca-certificates
• cloud-init
• cloud-guest-utils
• cloud-image-utils
• growpart
• cloud-initramfs-growroot
• open-iscsi
• openssh-server
• open-vm-tools



sudo apt-get update
sudo apt-get install -y curl wget git net-tools unzip ca-certificates cloud-init cloud-guest-utils cloud-image-utils cloud-initramfs-growroot open-iscsi openssh-server open-vm-tools net-tools apparmor

2. Configure the datasource for cloud-init in the template VM.

Rancher will use cloud-init for things like setting hostname, creating a user, running a script, etc.
Set the datasource for cloud-init using command “dpkg-reconfigure cloud-init”.



sudo dpkg-reconfigure cloud-init

And ensure “NoCloud” datasource is selected like below, I have deselected all other datasources since my requirement for rancher is only “NoCloud”.

Verify that changes are propagated to the config file,



root@:~# cat /etc/cloud/cloud.cfg.d/90_dpkg.cfg
# to update this file, run dpkg-reconfigure cloud-init
datasource_list: [ NoCloud ]
root@:~#

3. Convert the VM into a template.

Run below script to scrub the VM ( similar to sysprep on Windows ).
Save below contents to a file and execute it.



#!/bin/bash
# Cleaning logs.
if [ -f /var/log/audit/audit.log ]; then
  cat /dev/null > /var/log/audit/audit.log
fi
if [ -f /var/log/wtmp ]; then
  cat /dev/null > /var/log/wtmp
fi
if [ -f /var/log/lastlog ]; then
  cat /dev/null > /var/log/lastlog
fi

# Cleaning udev rules.
if [ -f /etc/udev/rules.d/70-persistent-net.rules ]; then
  rm /etc/udev/rules.d/70-persistent-net.rules
fi

# Cleaning the /tmp directories
rm -rf /tmp/*
rm -rf /var/tmp/*

# Cleaning the SSH host keys
rm -f /etc/ssh/ssh_host_*

# Cleaning the machine-id
truncate -s 0 /etc/machine-id
rm /var/lib/dbus/machine-id
ln -s /etc/machine-id /var/lib/dbus/machine-id

# Cleaning the shell history
unset HISTFILE
history -cw
echo > ~/.bash_history
rm -fr /root/.bash_history

# Truncating hostname, hosts, resolv.conf and setting hostname to localhost
truncate -s 0 /etc/{hostname,hosts,resolv.conf}
hostnamectl set-hostname localhost

# Clean cloud-init
cloud-init clean -s -l

Now clone the VM to a template.

4. Add vSphere vCenter credentials in Rancher.

Click Create and Click on VMware vSphere

Enter your vSphere vCenter credentials, we will use administrator account. For granular permissions, please refer rancher documentation.

5. Create RKE2 Cluster in vSphere

Go back to Rancher homepage, Click on Create.

Ensure you toggle the switch to RKE2 as highlighted below and click on VMware vSphere.

Enter the details,

Pool1, we will use to create "Control Plane Nodes". Ensure you select appropriate Data Center/Resource Pool/Data Store/Folder.

Select the Template that you created in the initial steps, along with CPU/Memory/Networks/etc like below.

Add another pool for worker node,

Fill the information like we did above for control plane nodes.

For the sake simplicity, we will keep the default values for the cluster and Click Create,

Cluster creation in progress,

Wait for the nodes to be bootstrapped and cluster creation,

If you see VM settings, rancher would have mounted an iso called user-data.iso

If you login to one of the node and navigate to /mnt, it will have user-data and meta-data and used by cloud-init (if you remember, this is the reason, we selected NoCloud as data source for cloud-init)



root@vsphere-rke2-test01-pool1-4a86ea2d-tf5jq:~# cd /mnt

root@vsphere-rke2-test01-pool1-4a86ea2d-tf5jq:/mnt# ls

meta-data  user-data

root@vsphere-rke2-test01-pool1-4a86ea2d-tf5jq:/mnt# cat meta-data

hostname: vsphere-rke2-test01-pool1-4a86ea2d-tf5jq

root@vsphere-rke2-test01-pool1-4a86ea2d-tf5jq:/mnt#

root@vsphere-rke2-test01-pool1-4a86ea2d-tf5jq:/mnt# cat user-data

  
  
  cloud-config


groups:


staff
hostname: vsphere-rke2-test01-pool1-4a86ea2d-tf5jq
runcmd:
sh /usr/local/custom_script/install.sh
set_hostname:
vsphere-rke2-test01-pool1-4a86ea2d-tf5jq
users:
create_groups: false
groups: staff
lock_passwd: true
name: docker
no_user_group: true