How can I read variables into Terraform from a YAML file to deploy the resources on the AWS Cloud?

Hello everyone!

I've been joining a IaC project (Infrastructure as Code) where the Terraform has been used to deploy our resources into the Cloud, and some variables are being to store in the YAML file. Participating on the project, I have noticed that this model has some advantages as you can see below:

- Readability: the YAML use more human-readable syntax which it makes easier to understand each varaibles and their values.

- Organization: as the YAML permit to organize the varaibles into groups, you can manage it better in a complex infrastructure with a lot the varaibles.

- Better unification: YAML is much used with other DevOps tools, as for example Ansible, then it become esier to share the varaibles with different tools.

So, if you use the YAML file to store the Terraform variables, it will offer more flexibility and scalability to manage your infrastructure as code.

Objective:

Here I'm going to using a public module from the Terraform registry to deploy a VPC network and subnets (private and public) on AWS cloud, and the Terraform varaibles will become store into YAML file. It'll be an example, but feel free to use it to deploy other resources.

Note: In this post, I will not comment how the Terraform and module works, or how the AWS network resource works. If desired, comment here because I can develop another post or share AWS documentation explaining about each resource.

Before start, there are some requirements to deploy this scenarios that will be expose here. It's needed to have the Terraform and AWS CLI installed, AWS account set, and in my case, I'm going to use the VScode as source-code editor.

So, let's get started?

Firstly, let's clone the my public terraform_study github repository where there are the terraform code stored. So you can use the the git clone command to download the Terraform codes or feel free to copy manually. Here we're going to use the git clone command in my Linux machine to become easier.

cd ~
git clone https://github.com/hugothomazpsouza/terraform_study.git
cd terraform_study/Deploy_VPC_and_Subnets_resources_with_Variables_store_on_the_YAML_file/
ls -la

Result:

After completed the commands, you will se the some Terraform files and one YAML file.

Let's get start to talk about the "locals.tf" Terraform file. It refer to local variables in Terraform, and the tip here is becasue I'm using the "yamldecode" function to read "variables.yaml" YAML file as input for Terraform.

Now, let's check the "variables.yaml" YAML file. This YAML file defines two VPCs: "vpc_core_network" and "vpc_app_network", and each one store the VPC CIDR block, Availability Zones, Private and Public Subnets varaibles.

As defined on this file, we're going to create two VPCs, 3 Private subnets each one in an Availability Zone and also 3 Public subnets each one in an Availability Zone.

The "vpc.tf" Terraform file, as I said before, I'm calling a public module from the Terraform registry to deploy a VPC on AWS Cloud, but I've adjusted the input variables to get from YAML file, and I've created for_each argument for setting to the value of local.config.vpcs, meaning that Terraform will run the module once for each key-value pair in this data structure.

Now, we're going to talk about of other three Terraform files.

outputs.tf - it is used to define the outputs variables in Terraform. The outputs values can be used in other Terraform configurations or scripts after the resources has been deployed. For example, if you would like to get the VPC_ID to create another resource you can get it through the Outputs variables instead of set the VPC_ID manually. It will become your code more dynamically and scalable.

provider.tf - It is used to configure the Terraform provider that is used, in this case we've used AWS provider, but we can use other Cloud provider, such as Google Cloud, and other. On this provider file I've also defined the authentication method and the configuration data needed to access the provider.

version.tf - It is used to specify the version of Terraform configuration syntax. It is recommended to specify the Terraform version to ensure that Terraform uses the correct syntax for the desired version.

Now, after completed the understand about the Terraform files and YAML file, we're going to deploy the resources.

Through the CLI, let's run the Terraform init command to load the modul, installs the necessary providers, and also sets the backend for storing the Terraform state file.

After that, let's run the Terraform plan command to see the changes that the Terraform will make to your infrastructure. I'm going to add the command below, but I won't show the output of the command because there will be many lines.

You will see how many resources will add, change or destroy.

And now, let's run Terraform apply to deploy the resources shown in the terraform plan command.

There will be a question to approve the deployment, then type "yes".

After completed it, you will see the information how many resources were added, changed or destroyed, and also the outputs variables.

Now, you can go to the AWS console to see the resources deployed.

Note: For deleting all resources deployed, please, run the Terraform destroy command. You will need to approve it, then type "yes".

Conclusion:

The idea here was shown how to read variables into Terraform from a YAML file to deploy the resources in the AWS Cloud. This was a tasting, but you reuse this idea to use according to your environments.

Well, I hope you enjoyed it!