Terraform Cloud Project Bootcamp with Andrew Brown - Storing the state file in Terraform Cloud

Hi guys , in this piece we will learn how to store the terraform state file that is created when we run terraform apply to Terraform Cloud.

This article is part of my Terraform journey with Terraform Bootcamp by Andrew Brown and Andrew Bayko, together with Chris Williams(I am also using his resources that he published here and the beloved Shala Warner. I am also using some other resources from Aaron Brooks so that I can better explain new terms. And a special shout out to Gwen Leigh for such a helpful outline that I used as a guide to create this series so that the documentation is easy to read!

As I learn more about Terraform, feel free to follow Andrew Brown on Youtube and see their free (and paid) content . Now let's jump in.

Table of Contents

• Persisiting our local state file in Terraform Cloud
  • What are Terraform State files?
• Create a Terraform Cloud account
  • 1. We will create a new project called terraform-beginner-bootcamp2023.
  • 2. Create a new workspace within the project.
  • 3. Run terraform init
  • 4. Generating our terraform state file
  • 5. Choose our preferred workflow
  • 6. In main.tf file, we will edit it as below
• Automating the creation of the terraform login
• Resources

Persisiting our local state file in Terraform Cloud

What are Terraform State files?

The statefile can contain a lot of secrets!!!

• resource IDs
• DB username/passwords
• private keys

So you should treat it like you would your company passwords. By default local state is stored in plaintext as JSON. You can mark sensitive information in your config files as such with the sensitive = true argument.

redacted from the CLI output, but still is in the statefile as plaintext (that's why it's important to lock down the statefile).

Use a backend that encrypts and protects your statefile from unauthorized access.

TFC encrypts state at rest & in transit
Turn on encryption if you are using S3 (& use state locking!)
Terraform does not persist state to local disk when remote state is being used

Create a Terraform Cloud account

We will need to create a Terraform Cloud account here

A Terraform project can have multiple workspaces, and a workspace is a configuration for an infrastructure project.

1. We will create a new project called terraform-beginner-bootcamp2023.

2. Create a new workspace within the project.

3. Run terraform init

We have to run this multiple times because our Gitpod environment is restarted every time we shut down. We dont have to do this if we are doing this project on a local machine/environment.

4. Generating our terraform state file

To generate our terraform state file terraform.tfstate, we will run terraform apply --auto-approve.

Since Terraform is agnostic, we can store our statefile in S3, locally or on Terraform Cloud.

5. Choose our preferred workflow

We will choose our preferred workflow in Terraform Cloud, which is the CLI-driven workflow, name the workspace, terrahouse-1. and add in the description. Create the workspace and. The organization will appear as below, with the name and email address already filled in

6. In main.tf file, we will edit it as below

terraform {
  cloud {
    organization = "msaghu"

    workspaces {
      name = "terra-house-renaissance"
    }
  }

  required_providers {
    random = {
      source = "hashicorp/random"
      version = "3.5.1"
    }
    aws = {
      source = "hashicorp/aws"
      version = "5.19.0"
    }
  }
}

provider "aws" {
  # Configuration options
}

provider "random" {
  # Configuration options
}

#https://registry.terraform.io/providers/hashicorp/random/latest/docs/resources/string
resource "random_string" "bucket_name" {
  lower = true
  upper = false
  length           = 32
  special          = false
}

#Terraform AWS S3 
#https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket
resource "aws_s3_bucket" "example" {
  #bUCKET NAMING RULES
  #https://docs.aws.amazon.com/AmazonS3/latest/userguide/bucketnamingrules.html
  bucket = random_string.bucket_name.result
}

output "random_bucket_name" {
  value = random_string.bucket_name.result
}

• We have the organization as the organization name we made when creating a Terraform Cloud

  backend "remote" {
    hostname = "app.terraform.io"
    organization = "company"

    workspaces {
      name = "my-app-prod"
    }

• Establish a connection to Terraform cloud, run; terraform login and enter yes when prompted

In the Terraform Cloud page, go to the Tokens tab and copy the token provided.

Create a file in Gitpod touch /home/gitpod/.terraform.d/credentials.tfrc.json

Open the file open /home/gitpod/.terraform.d/credentials.tfrc.json and add in the following:

{
  "credentials": {
    "app.terraform.io": {
      "token": "YOUR_TERRAFORMCLOUD_TOKEN//PASTE IN THE CODE ABOV"
    }
  }
}

Save and close the file

• Run terraform init which will tell you that it wants to copy the terraform.tfstate file to Terraform. When prompted enter yes

Go to Terraform cloud and see that the resources were created when we ran terraform init, plan and apply, i.e the S3 bucket and its random name, can now be seen on the Terraform cloud workspace.

Automating the creation of the terraform login

If we are using a workspace like Gitpod, we will need to do this step. Find the code here
We need to create credentials.tfrc.json;

• In Terraform Cloud, create a user token for about 30 days.

• Add the token as an environment variable in Gitpod by adding in the terminal;
  gp env TERRAFORM_CLOUD_TOKEN='usertokenfor30daysusertokenfor30daysusertokenfor30daysusertokenfor30days'

then
export TERRAFORM_CLOUD_TOKEN='usertokenfor30daysusertokenfor30daysusertokenfor30daysusertokenfor30days'

• We will create a bash file that will generate the tfrc credentials in bin folder ; bin/generate_tfrc_credentials

#!/usr/bin/env bash

# Define the target directory
TARGET_DIR="/home/gitpod/.terraform.d"
TARGET_FILE="${TARGET_DIR}/credentials.tfrc.json"

# Check if the TERRAFORM_CLOUD_TOKEN environment variable is set
if [ -z "$TERRAFORM_CLOUD_TOKEN" ]; then
  echo "Error: TERRAFORM_CLOUD_TOKEN environment variable is not set."
  exit 1
fi

# Create the directory if it doesn't exist
if [ ! -d "$TARGET_DIR" ]; then
  mkdir -p "$TARGET_DIR"
fi

# JSON structure for credentials.tfrc.json
json_data='{
  "credentials": {
    "app.terraform.io": {
      "token": "'"$TERRAFORM_CLOUD_TOKEN"'"
    }
  }
}'

# Write the JSON data to credentials.tfrc.json
echo "$json_data" > "$TARGET_FILE"

echo "$TARGET_FILE has been generated."

• Change the file permissions by 'chmod u+x .bin/generate_tfrc_credentials

Make sure to note the Terraform 30 day token somewhere but DO NOT commit the file.

Resources

1. Migrating state to Terraform Cloud
2. Terraform backend remote
3. Terraform Certified Associate (003) by Chris Williams