Input Variables File for Multiple Terraform Workspaces

We've all been there: you have written a Terraform configuration that is so awesome, you have to apply it multiple times... Or at least to dev, test and prod. However amazing you configuration might be, it's very rare to have all details exactly the same over every environment. You usually end up with something like this, and that's perfectly fine (for readability, I've condensed the code a bit, see this gist for a more complete example):

resource "aws_instance" "this" {
  count = var.instance_count

  ami           = var.ami_id
  instance_type = var.instance_type

  tags = {
    Name = "project-xyz-${terraform.workspace}-${count.index}"
  }
}

Where this starts to hurt is during apply time, because you will have for example:

terraform workspace select prod
terraform apply -var='instance_count=5' -var='ami_id=ami-1a2b3c' -var='instance_type=t2.large'

This line of variables becomes unclear very fast. Also, remembering all those variables or making your command line remember the statement (per environment) is rather cumbersome. Moreover, making this work in automation (again, per environment...) will most likely require some env variables magic. This separates the definition of your infrastructure from the standard ways of calling it, which is bad. (By the way, if you have never heard of connascence, stop reading this article and read this one instead.)

Terraform Cloud offers some remediation to this, but also here you create a separation between template and env variable sets. It would be much better if we could keep the default sets of variables in version control next to our configuration. After some tinkering, I found the following solution:

variable "workspace_variables" {
  type = map(object({
    instance_count = number
    ami_id         = string
    instance_type  = string
  }))
  default = {}
}

locals {
  default_vars = lookup(var.workspace_variables, terraform.workspace, {
    instance_count = 2
    ami_id         = "ami-abc123"
    instance_type  = "t2.micro"
  })
  instance_count = var.instance_count != null ? var.instance_count : local.default_vars["instance_count"]
  ami_id         = var.ami_id != null ? var.ami_id : local.default_vars["ami_id"]
  instance_type  = var.instance_type != null ? var.instance_type : local.default_vars["instance_type"]
}

This code snippet reads one an optional variable to rule them all: workspace_variables. If none is provided, it will use the default sub-variable set. Also, you can still override any set of variables with the Terraform cli. Next to this snippet, you will need to make the default for all variables null and you will have to replace var. with local. when using them. Then you can create a Terraform vars file with something like this:

workspace_variables = {
  dev = {
    instance_count = 2
    ami_id         = "ami-abc123"
    instance_type  = "t2.micro"
  }
  test = {
    instance_count = 3
    ami_id         = "ami-a1b2c3"
    instance_type  = "t2.medium"
  }
  prod = {
    instance_count = 5
    ami_id         = "ami-1a2b3c"
    instance_type  = "t2.large"
  }
}

Much clearer and all kept in one findable place that can be put in version control! And you will get the configuration you want with a simple Terraform flow of selecting your workspace and hitting apply. Also, you can still override the variables with the command line for hot fixes, tests etc.:

terraform workspace select prod
terraform apply -var='ami_id=ami-x1337x'

It may look like too much boilerplate for simple parameter input, but I've found that it weighs up to setting variables the 'normal' way if there are a lot of them. See if it works for you!

Final remark: it would require some more tinkering and boilerplate to implement proper Terraform variable validation and forward compatibility of the vars file, but that might be fixed when optional variable attributes comes out of public beta.

(The code above is condensed for readability, for a more complete example see this gist.)