Spacelift team for Spacelift

Posted on Jul 26 • Originally published at spacelift.io

Using Terraform YAML Functions

#terraform #devops #infrastructureascode

In this article, we will take a look at two functions used in Terraform to manipulate YAML: yamlencode and yamldecode .

First, we will look at how YAML can be used with Terraform before describing the functions and how they map Terraform types to YAML. We will then show example usage and code for both functions, showing how to use them in practice. Let's go!

What is YAML?

YAML (YAML Ain't Markup Language) is a human-readable data serialization format. It stands out for its simplicity and readability, making it popular for configuration files, data exchange between programming languages, and other similar use cases. YAML has a minimalistic syntax compared to other data serialization formats like JSON or XML. Despite its simplicity, YAML can represent complex data structures and relationships.

YAML is often used in contexts where data needs to be structured hierarchically, such as configuration files for software applications, data storage, and communication protocols, and specifically more common in infrastructure configuration files for Docker, Kubernetes, Ansible, and APIs. It uses indentation to represent nesting, similar to the way Python does. YAML is designed to be language-agnostic, meaning it can be used with virtually any programming language. Its human-readable nature and ease of use have contributed to its widespread adoption in the software development community.

Can YAML be used in Terraform?

Terraform primarily uses its own declarative language called HashiCorp Configuration Language (HCL) for defining infrastructure as code. Configuration files cannot be written in YAML directly.

Terraform has yamlencode and yamldecode functions available, which allow you to convert values between Terraform and YAML format. These functions are the main focus of this article.

Alternatively, Terraform does support JSON as an alternative input format, and YAML files can be converted to JSON. There are various tools and libraries available for converting YAML to JSON in different programming languages. For example, you can use Python with libraries like pyyaml or ruamel.yaml to parse YAML and then output JSON.

What is the yamldecode function in Terraform?

The yamldecode is a Terraform function used to parse a string formatted in a subset of YAML 1.2 and convert it into a Terraform data structure. It allows you to decode YAML data into Terraform's data types, such as strings, numbers, booleans, lists, maps, etc.

Below is a simple example of using the yamldecode() function:

> yamldecode("hello: world")
{
 "hello" = "world"
}

How yamldecode maps Terraform types to YAML?

yamldecode supports a limited set of YAML features to ensure compatibility with Terraform's data types. Terraform automatically converts the parsed YAML data to the most appropriate Terraform type (e.g., string, number, list, or object).

Note that the function doesn't support the full YAML specification. For instance, it cannot handle cyclic data structures or tags beyond a specific set. Because of these limitations, using yamldecode followed by yamlencode (which encodes Terraform data back to YAML) might not produce an identical YAML output so be sure to test whilst implementing your code.

Basic YAML types

String - Plain strings in YAML directly map to Terraform strings.
Number - Integers and floating-point numbers in YAML become Terraform numbers.
Boolean - true and false in YAML translate to Terraform booleans.
Lists - YAML lists become Terraform lists. The elements within the list will be converted based on their individual data types.
Maps (Objects) - YAML maps (represented by key-value pairs) are converted to Terraform maps. Keys in YAML maps must be strings, which become the keys in the Terraform map. Values in the map are converted based on their data type in YAML.

Examples: Using the Terraform yamldecode function

Let's take a look at some practical examples of retrieving values from YAML files using the yamldecode function in Terraform

1. Getting the API key from a YAML file

In the first example, we have a config.yaml file containing an API key that we want Terraform to be able to read. It is then outputted using the local-exec provisioner.

# YAML file (config.yaml)
api_key: "your_api_key_here"

# Terraform code (test.tf)
api_key = yamldecode(file("config.yaml"))["api_key"]

resource "null_resource" "api_key_usage" {
 provisioner "local-exec" {
   command = "echo Using API key: ${var.api_key}"
 }
}

2. Accessing nested data in a YAML file

Taking this a step further, the next example shows how to access nested data within the YAML configuration.

# YAML file (server_config.yaml)
servers:
 web:
   host: "webserver.example.com"
   port: 80
 api:
   host: "api.example.com"
   port: 443

# Terraform code
server_config = yamldecode(file("server_config.yaml"))

resource "null_resource" "web_server_info" {
 provisioner "local-exec" {
   command = "echo Web server: ${server_config.servers.web.host}:${server_config.servers.web.port}"
 }
}

resource "null_resource" "api_server_info" {
 provisioner "local-exec" {
   command = "echo API server: ${server_config.servers.api.host}:${server_config.servers.api.port}"
 }
}

3. Handling list of objects defined in a YAML file

The next example shows how to handle a list of objects that might be defined in your YAML file.

A loop iterates through the security_groups list using a for loop within a Terraform provisioner. Inside the loop, each element's properties (name and description) are accessed and used within the echo command.

# YAML file (security_groups.yaml)
security_groups:
 - name: "web-server-sg"
   description: "Security group for web server instances"
 - name: "database-sg"
   description: "Security group for database instances"

# Terraform code
security_group_config = yamldecode(file("security_groups.yaml"))

resource "null_resource" "security_group_info" {
 provisioner "local-exec" {
   command = <<EOF
     for group in var.security_group_config.security_groups :
       echo Name: ${group.name}, Description: ${group.description}
     done
EOF
 }
}

💡 You might also like:

What is the yamlencode function in Terraform?

The opposite of yamldecode, the yamlencode function takes a Terraform value (such as a map or a list) and returns a string representing the YAML-encoded version of that value. This string can then be used within your Terraform configuration to generate YAML files.

Input: It accepts any valid Terraform data type (strings, numbers, booleans, lists, or maps).
Output: It returns a string representing the data in YAML format.

How to generate YAML from a Terraform template file

If you have a Terraform template file with placeholders for dynamic values, you can use the templatefile function to render the template with actual data and then use yamlencode to convert the rendered content into a YAML string.

Here's a breakdown of the steps:

Template File:

Create a file (e.g., config.tmpl) with placeholders for your desired configuration values.
Use Terraform interpolation syntax (var.name) for the placeholders.

Terraform Code:

Define variables for the dynamic values you want to inject into the template.
Use the templatefile function to read the template and replace placeholders with actual values:

In this example, the local-exec provisioner writes the generated YAML string (yaml_config) to a file named config.yaml.

template_content = templatefile(file("config.tmpl"), {
 name = var.config_name
 # Add more variables and interpolation as needed
})

yaml_config = yamlencode(template_content)

resource "null_resource" "write_yaml" {
 provisioner "local-exec" {
   command = <<EOF
     echo "${yaml_config}" > config.yaml
EOF
 }
}

Examples: Using Terraform yamlencode function

Now, we'll show some practical examples for using the yamlencode function in Terraform.

1. Generating a simple YAML configuration with dynamic values

This example demonstrates how to generate a simple YAML configuration with dynamic values from Terraform variables. The yamlencode function takes a map as input, where keys become YAML keys, and values are converted to their corresponding YAML representation.

# Terraform code

variable "app_name" {
 type = string
}

variable "server_port" {
 type = number
}

# Generate YAML using variables
yaml_config = yamlencode({
 name: var.app_name
 port: var.server_port
})

resource "null_resource" "write_yaml" {
 provisioner "local-exec" {
   command = <<EOF
     echo "${yaml_config}" > config.yaml
EOF
 }
}

2. Creating a YAML configuration for a user with SSH keys

In this example, we create a YAML configuration for a user with SSH keys. The ssh_keys variable is a list of strings, which gets converted to a YAML list during encoding.

# Terraform code

variable "user_name" {
 type = string
}

variable "ssh_keys" {
 type = list(string)
}

# Generate YAML with nested data and a list
user_config = yamlencode({
 username: var.user_name
 ssh_keys: var.ssh_keys
})

resource "null_resource" "write_user_config" {
 provisioner "local-exec" {
   command = <<EOF
     echo "${user_config}" > user_config.yaml
EOF
 }
}

3. Handling nested data structures in Terraform

This example shows how to handle nested data structures in Terraform. The web_servers variable holds a list of objects, each representing a web server with its host and port. yamlencode recursively encodes this nested structure into the corresponding YAML format.

# Terraform code

variable "web_servers" {
 type = list(object({
   host = string
   port = number
 }))
}

# Generate YAML with nested objects in a list
server_config = yamlencode({
 servers: var.web_servers
})

resource "null_resource" "write_server_config" {
 provisioner "local-exec" {
   command = <<EOF
     echo "${server_config}" > servers.yaml
EOF
 }
}

Key points

To read and output YAML in your Terraform configurations, you can use the built-in functions yamlencode and yamldecode

yamldecode parses a string as a subset of YAML, and produces a representation of its value.yamlencode performs the opposite operation, encoding a given value to a string using YAML 1.2 block syntax.

We encourage you also to explore how Spacelift makes it easy to work with Terraform. If you need any help managing your Terraform infrastructure, building more complex workflows based on Terraform, and managing AWS credentials per run, instead of using a static pair on your local machine, Spacelift is a fantastic tool for this. If you want to learn more, create a free account today or book a demo with one of our engineers.

Written by Jack Roper

DEV Community