DEV Community

Cover image for Creating proxmox templates with packer
Aaron Berry
Aaron Berry

Posted on • Edited on

Creating proxmox templates with packer

I've been using proxmox for a while now in my homelab as an open-source alternative to something like ESXi for a virtualization platform. With proxmox when can create templates for our VMS so we can have a standard starting point to install our applications on top of, these templates can be useful too so that you can pre-install packages for authentication, security, logging and etc without anyone else needing to think about it.

However, creating and managing these templates can become a challenge with how time-consuming and manual it can be. I want to show you how you can make this process more standardized and automated with the use of packer to allow you to declare your proxmox templates as code.

What is packer

Packer is a utility that allows you to build virtual machine images so that you can define a golden image as code. Packer can be used to create images for almost all of the big cloud providers such as AWS, GCE, Azure and Digital Ocean, or can be used with locally installed hypervisors such as VMWare, Proxmox and a few others.

To build an image with packer we need to define our image through a template file. The file uses the JSON format and comprises of 3 main sections that are used to define and prepare your image.

  • Builders: Components of Packer that are able to create a machine image for a single platform. A builder is invoked as part of a build in order to create the actual resulting images.

  • Provisioners: Install and configure software within a running machine prior to that machine being turned into a static image. Example provisioners include shell scripts, Chef, Puppet, etc.

  • Post Processors: Take the result of a builder or another post-processor and process that to create a new artifact. Examples of post-processors are compress and upload to compress and upload artifacts respectively, etc.

By using packer we can define our golden VM image as code so that we can easily build identically configured images on demand so that all your machines are running the same image and can also be easily updated to a new image when needed.

Preparing your packer template

For the following examples ill be referencing files from this example repo you can follow along with and should be a solid template to start off from for your own custom proxmox templates.

To create the template we will use the proxmox builder which connects through the proxmox web API to provision and configure the VM for us and then turn it into a template. To configure our template we will use a variables file, to import this variables file we will use the -var-file flag to pass in our variables to packer. These variables will be used in our template file with the following syntax within a string like so "passwd/username={{ user 'ssh_username'}}".

Now the builder block below will outline the basic properties of our desired proxmox template such as its name, the allocated resources and the devices attached to the VM. To achieve this the boot_command option will be used to boot the OS and tell it to look for the preseed file to automate the OS installation process. Packer has an inbuilt HTTP server to serve this preseed.cfg file to the VM as its installing by using the http_directory option in the builder to specify the public files of the HTTP server. Check out the ubuntu preseed documentation for info on modifying the automatic installation process of the OS via a pre seed file.

{
  "builders": [
    {
      "type": "proxmox",
      "proxmox_url": "https://{{user `proxmox_host`}}:8006/api2/json",
      "insecure_skip_tls_verify": true,
      "username": "{{user `proxmox_api_user`}}",
      "password": "{{user `proxmox_api_password`}}",

      "vm_id": "{{ user `vmid` }}",
      "vm_name": "{{user `template_name`}}",
      "template_description": "{{ user `template_description` }}",

      "node": "{{user `proxmox_node`}}",
      "cores": "{{ user `cores` }}",
      "sockets": "{{ user `sockets` }}",
      "memory": "{{ user `memory` }}",
      "os": "l26",

      "network_adapters": [
        {
          "model": "virtio",
          "bridge": "vmbr0"
        }
      ],

      "disks": [
        {
          "type": "scsi",
          "disk_size": "{{ user `disk_size`}}",
          "storage_pool": "{{user `datastore`}}",
          "storage_pool_type": "{{user `datastore_type`}}",
          "format": "raw",
          "cache_mode": "writeback"
        }
      ],

      "ssh_timeout": "90m",
      "ssh_password": "{{ user `ssh_password` }}",
      "ssh_username": "{{ user `ssh_username` }}",

      "qemu_agent": true,
      "unmount_iso": true,

      "iso_file": "{{user `iso`}}",

      "http_directory": "./http",

      "boot_wait": "10s",

      "boot_command": [
        "{{ user `boot_command_prefix` }}",
        "/install/vmlinuz ",
        "auto ",
        "console-setup/ask_detect=false ",
        "debconf/frontend=noninteractive ",
        "debian-installer={{ user `locale` }} ",
        "hostname={{ user `hostname` }} ",
        "fb=false ",
        "grub-installer/bootdev=/dev/sda<wait> ",
        "initrd=/install/initrd.gz ",
        "kbd-chooser/method=us ",
        "keyboard-configuration/modelcode=SKIP ",
        "locale={{ user `locale` }} ",
        "noapic ",
        "passwd/username={{ user `ssh_username` }} ",
        "passwd/user-fullname={{ user `ssh_fullname` }} ",
        "passwd/user-password={{ user `ssh_password` }} ",
        "passwd/user-password-again={{ user `ssh_password` }} ",
        "preseed/url=http://{{ .HTTPIP }}:{{ .HTTPPort }}/{{ user `preseed.cfg` }} ",
        "-- <enter>"
      ]
    }
  ]
}
Enter fullscreen mode Exit fullscreen mode

In this template, we will also be using the shell provisioner to configure our VM os once it has been installed onto the virtual machine and is available via SSH. This can be helpful for installing the minimum required packages on your VM's such as the QEMU quest agent and cloud init or any other software required. You can also switch this provisioner auto for any of the other provisioners such as ansible, chef or puppet for example.

{
  "provisioners": [
    {
      "pause_before": "20s",
      "type": "shell",
      "environment_vars": ["DEBIAN_FRONTEND=noninteractive"],
      "inline": [
        "date > provision.txt",
        "sudo apt-get update",
        "sudo apt-get -y upgrade",
        "sudo apt-get -y dist-upgrade",
        "sudo apt-get -y install linux-generic linux-headers-generic linux-image-generic",
        "sudo apt-get -y install qemu-guest-agent cloud-init",
        "sudo apt-get -y install wget curl",
        "exit 0"
      ]
    }
  ]
}
Enter fullscreen mode Exit fullscreen mode

And finally, we will use the post processors to run some commands locally. This will make an SSH connection to the PVE host and run some commands manually to set up the virtual devices necessary for cloud init. This post-processor is using the shell-local post processor to run the commands on the local machine running packer but you could always move this configuration to something like an ansible playbook to make the configuration more readable and portable.

{
  "post-processors": [
    {
      "type": "shell-local",
      "inline": [
        "ssh root@{{user `proxmox_host`}} qm set {{user `vmid`}} --scsihw virtio-scsi-pci",
        "ssh root@{{user `proxmox_host`}} qm set {{user `vmid`}} --ide2 {{user `datastore`}}:cloudinit",
        "ssh root@{{user `proxmox_host`}} qm set {{user `vmid`}} --boot c --bootdisk scsi0",
        "ssh root@{{user `proxmox_host`}} qm set {{user `vmid`}} --ciuser {{ user `ssh_username` }}",
        "ssh root@{{user `proxmox_host`}} qm set {{user `vmid`}} --cipassword {{ user `ssh_password` }}",
        "ssh root@{{user `proxmox_host`}} qm set {{user `vmid`}} --vga std"
      ]
    }
  ]
}
Enter fullscreen mode Exit fullscreen mode

Now with your configuration complete, you will be ready to build your proxmox template with packer. Run the command packer build -var-file="./config.json" ./ubuntu-18.04.json You should see some output for each of the builders, provisioners and post-processors.

$ packer build -var-file="./config.json" ./ubuntu-18.04.json
proxmox: output will be in this color.

==> proxmox: Creating VM
==> proxmox: Starting VM
==> proxmox: Starting HTTP server on port 8771

...

Build 'proxmox' finished.

==> Builds finished. The artifacts of successful builds are:
--> proxmox: A template was created: 4444
--> proxmox:
Enter fullscreen mode Exit fullscreen mode

When the process is complete you should see your template ready in the proxmox interface and ready to be cloned into virtual machines.

Completed proxmox template

Further reading on packer

You should now have a good starting point for building proxmox templates with packer. If your looking to extend its usefulness a little further check out these useful articles.

Connect further


Cover photo by chuttersnap on Unsplash

Top comments (5)

Collapse
 
lethargosapatheia profile image
lethargosapatheia • Edited

I don't think this applies anymore. Ubuntu 20.04 switched to subquity and gave up debian-installer (so preseeding). You're supposed to be using cloud-init directly. That doesn't work for me either, but this is what they announced quite a long time ago (since 2019).

Collapse
 
aaronktberry profile image
Aaron Berry

Yep that’s right cloud unit is the way to do with the newer distributions. I’ve been meaning to spend some time to do another write up for what that looks like with cloud unit but unfortunately I had to tear down my proxmox lab during a recent move 😬

Collapse
 
lethargosapatheia profile image
lethargosapatheia • Edited

I was able to solve it, actually, up to the point where you have a readily available template with cloud-init ready to read the user-data. Just to give you a head start :)

       "boot_command": [
        "<esc><enter><f6><esc><wait> ",
        "<bs><bs><bs><bs><bs>",
        "ip={{ user `vm_ip` }}::{{ user `vm_gateway` }}:{{ user `vm_netmask` }}::::{{ user `vm_dns` }} ",
        "autoinstall ds=nocloud-net;s=http://{{ .HTTPIP }}:{{ .HTTPPort }}/ ",
        "--- <enter>"
      ]
Enter fullscreen mode Exit fullscreen mode

This configures autoinstall, which can run cloud-init commands. And will run cloud-init anyway.
These are the variables:

  "variables": {
    "vm_ip": "10.0.0.159",
    "vm_netmask": "255.255.255.0",
    "vm_gateway": "10.0.0.126",
    "vm_dns": "10.0.0.126",
    "search_domain:": "domain.internal",
    "dns_server:": "1.1.1.1",
    "template_name": "ubuntu-20-04.domain.internal",
    "template_description": "Ubuntu 20.04, generated by packer on {{ isotime }}",
    "playbook": "setup/work.yml",
Enter fullscreen mode Exit fullscreen mode

Then you can reset cloud-init through ansible (run by packer), for example.

If you want to write an updated article, I'll be happy to help, if you think you need any, of course!

Collapse
 
mike1237 profile image
mike1237

I wrote a script to create a base template on Proxmox that supports cloud-init.
Then you can use proxmox-clone packer builder off of that.
dev.to/mike1237/create-proxmox-clo...

Collapse
 
jibnight profile image
jibnight

Thank you from France :)