How To Manage Secrets In Terraform Like A Pro

Managing secrets when provisioning infrastructure with terraform can be quite daunting. You have to think of ways to securely include your secret into your terraform configuration without necessarily exposing it externally.

Let's say you have a database to provision with terraform. How do you provide the credentials for your database without including them in source code and at the same time ensuring that the automated flow of terraform is not interrupted?

In reality, there are quite a number of options available to you to securely store sensitive data when writing terraform code.

They can include:

👉 Using Environmental Variables
👉 Encrypting Secrets (KMS, SOPS)
👉 Using Cloud Services (AWS secret manager)

1. - Environmental Variables

Environmental variables can be used to keep plain-text secrets out of your code by taking advantage of terraform's native support for environmental variables.
By simply using the TF_VAR_var feature of terraform, we can provide sensitive data without making them an intrinsic part of source code.

For example,to assign a password value of to our database using terraform:

# create RDS Postgres Instance
resource "aws_db_instance" "mydb" {
    db_name = "mydb"
    engine = "postgres"
    engine_version = "15"
    instance_class = "db.t4g.micro"
    allocated_storage = 10

    publicly_accessible = false
    skip_final_snapshot = true
    db_subnet_group_name = aws_db_subnet_group.private.name 

    username = "var.username"
    password = var.password

     depends_on = [ 
        aws_db_subnet_group.private
     ]
}

We can then provide the password value by executing the following Linux command

export TF_VAR_password="mypass123"
export TF_VAR_username="root"

This saves us the risk of having to include the value in source code or carelessly exposing this value.
What's more? you can prevent this command from showing up in the history command by properly configuring the Histcontrol environmental variable. When this is set correctly, commands that start with a space do not show up in the history. This way, we can execute the export command by preceding it with a space and thus prevent our secret from being exposed.

Now if you decide to use environmental variables, you must have a strategy for storing and managing your secrets. The most popular way of doing this is by using password managers.
Password managers like LastPass or 1pass are efficient ways of storing and managing your secrets.

Advantages of this approach

• It is an easy-to-use solution.
• It integrates well with password managers (pass, lastpass 1pass etc).
• It keeps sensitive data out of your source code.

Disdvantages of this approach

• It defeats automation by inserting manual approaches into the terraform workflow.
• It is difficult to use in a collaborative environment.
• It is quite primitive and provides no security guarantees.

2. - Encrypting Secrets (KMS, SOPS)

This is a more advanced way of managing secrets in terraform.
In this approach, we create a file containing the secrets. This file is then encrypted and checked into version control along with our terraform configuration.

2a. KMS
To use AWS KMS to manage secrets:

Step 1. Create an AWS Customer Managed Key

NB: Make sure that your terraform user has sufficient IAM permissions to invoke this key.

Step 2. Use the AWS KMS encrypt command to encrypt your credentails file.

aws kms encrypt \
    --key-id <YOUR-CMK-KEY-ID> \
    --plaintext fileb://credentials.yml \
    --output text \
    --query CiphertextBlob | base64 \
    --decode > credentails.yml.encrypted

This will encrypt our create an encrypted file called credentials.yml.encrypted which will contain our credentials. We can now safely delete the credentials.yml file. The credentials.yml.encrypted file is secure enough to be checked into version control.

Step 3. Use the aws_kms_secrets terraform data source to decrypt this secret. Use terraform locals to refer to the decrypted secret. Finally, update your resource to make use of this value.

# Use AWS KMS to decrypt database credentials
data "aws_kms_secrets" "db_credentials" {
  secret {
    # ... potentially other configuration ...
    name    = "db_creds"
    payload = filebase64("./db_creds.yml.encrypted")
  }
}

# Use locals to grab the decrypted KMS key
locals {
    db_creds = yamldecode(data.aws_kms_secrets.db_credentials.plaintext["db_creds"])
}

# Create RDS Postgress instance
resource "aws_db_instance" "mydb" {
    db_name = "mydb"
    engine = "postgres"
    engine_version = "15"
    instance_class = "db.t4g.micro"
    allocated_storage = 10

    publicly_accessible = false
    skip_final_snapshot = true
    db_subnet_group_name = aws_db_subnet_group.private.name 

    username = local.db_creds.username
    password = local.db_creds.password

    depends_on = [ 
        aws_db_subnet_group.private
     ]
}

2b. SOPS
SOPS - short for Secret Operation s - is an open-source text file editor that encrypts/decrypts files automatically.
SOPS can easily integrate with AWS KMS, GCP KMS, hashicorp vault etc.
Here is a great resource to learn more about SOPS.

This approach is by far a more elegant, efficient and secure method of adding secrets and other sensitive data while provisioning infrastructure with terraform.

3. - Using Cloud Services (AWS secret manager)

AWS Secrets Manager is a cloud service offered by AWS which can help to securely manage, retrieve, and rotate database credentials, API keys, and other secrets throughout their lifecycles.

To use Secret Manager in Terraform:

• Create Your secret

• Reference this secret in your resource configuration:

# Use data source to reference secrets manager secret
data "aws_secretsmanager_secret_version" "credentials" {
    secret_id = "db_creds-v2"
}

# Use locals to grab the decrypted key from secret manager
locals {
    db_credentials = jsondecode(
        data.aws_secretsmanager_secret_version.credentials.secret_string
    )
}

# Create RDS Postgress instance
resource "aws_db_instance" "mydb" {
    db_name = "mydb"
    engine = "postgres"
    engine_version = "15"
    instance_class = "db.t4g.micro"
    allocated_storage = 10

    publicly_accessible = false
    skip_final_snapshot = true
    db_subnet_group_name = aws_db_subnet_group.private.name 

    username = local.db_credentials.username
    password = local.db_credentials.password

    depends_on = [ 
        aws_db_subnet_group.private
     ]
}

Again, make sure that your terraform user has the proper IAM permissions.

One major advantage of this method over AWS KMS is that you do not need to create a credentials file at all. This way, you can always modify your secrets without ever editing the source code!

Using AWS Secrets Manager to handle sensitive data in terraform is an elegant, secure and reliable strategy.