My latest project has been to re-create my homelab using cloud native principles such as Infrastructure-as-Code and Dev(Git)Ops.
After much research and deliberation, I've concluded the best way for me to achieve this is to separate my local hardware from the software which will run on it. Essentially, I'll be treating my hardware as a "local cloud", and create VM Images that I will deploy running my application stack. (Azure Stack HCI has the right idea here!)
When developing and testing on cloud providers, they always have a base image available to use as a starting off point. Not so for my local cloud!
In this article, I'll demonstrate how to create a cloud-init enabled Ubuntu 20.04 LTS base image to use on Proxmox VE.
There are several guides out there which utilize the "proxmox-iso" packer builder to build a base image from an ISO file. Modern Linux distributions are increasingly moving away from this install method and preseed files. Rather, disk images are provided with the OS pre-installed, and configuration is performed via cloud-init. We will create a Proxmox KVM base image using Ubuntu's KVM cloud image.
NOTE: Since the Packer Proxmox builder doesn't support LXD images nor templates, we need to create a KVM template.
The Proxmox API doesn't appear to offer the full functionality provided by the native shell commands to create a template, so we will run a script via SSH.
This gist contains the script, and I'll step through it below. https://gist.github.com/mike1237/cce83a74f898b11c2cec911204568cf9
We are downloading the kvm disk image. Note that this is a qcow2 image format with an extension of .img
Promxox doesn't like this so we rename the disk image to .qcow2
SRC_IMG="https://cloud-images.ubuntu.com/focal/current/focal-server-cloudimg-amd64-disk-kvm.img" IMG_NAME="focal-server-cloudimg-amd64-disk-kvm.qcow2" wget -O $IMG_NAME $SRC_IMG
The Ubuntu 20.04 image we are going to use does not include the
qemu-guest-agent package which is needed for the Guest VM to report its IP details back to Proxmox. This is required for Packer to communicate with the VM after cloning. the template.
libguestfs-tools will allow us to embed
qemu-guest-agent into the image. You can also add any additional packages you'd like in your base image. Personally, I prefer to customize this base image later with packer so that the packages can live in source control.
apt update apt install -y libguestfs-tools virt-customize --install qemu-guest-agent -a $IMG_NAME
For best performance, virtio 'hardware' should be used. Additionally, cloud-init requires a serial console and cloudinit IDE (CDROM) drive. We will set the network config to DHCP so that we get an IP address. Lastly, we will expand the template disk image size so we have space to install items later. It appears packer doesn't support doing this later.
TEMPL_NAME="ubuntu2004-cloud" VMID="9000" MEM="512" DISK_SIZE="32G" DISK_STOR="local-lvm" NET_BRIDGE="vmbr0" qm create $VMID --name $TEMPL_NAME --memory $MEM --net0 virtio,bridge=$NET_BRIDGE qm importdisk $VMID $IMG_NAME $DISK_STOR qm set $VMID --scsihw virtio-scsi-pci --scsi0 $DISK_STOR:vm-$VMID-disk-0 qm set $VMID --ide2 $DISK_STOR:cloudinit qm set $VMID --boot c --bootdisk scsi0 qm set $VMID --serial0 socket --vga serial0 qm set $VMID --ipconfig0 ip=dhcp qm resize $VMID scsi0 $DISK_SIZE qm template $VMID # Remove downloaded image rm $IMG_NAME
Now that we have our cloud-init enabled image on Proxmox, we can use Packer to create a template based off of this template.
Ensure to set the
I'd recommend adding the Proxmox variables to a var file.
packer build --var-file=./proxmox.pkvars.hcl --var "proxox_template_name=test-output-template" --var "proxmox_source_template=ubuntu2004-cloud" base.pkr.hcl
Now that you've created a template using packer from the base template, you can use Terraform to deploy that VM!
I'm still working on that myself :)