DEV Community

Spacelift profile image Spacelift team
Spacelift team for Spacelift

Posted on • Originally published at spacelift.io

OpenTofu State File Encryption Overview

#opentofu #devops #infrastructureascode

State encryption has long been one of Terraform's most requested features, but it was never delivered. However, after just six months in existence, the OpenTofu team has listened to the community and released, state encryption.

OpenTofu's encryption mechanism allows you to encrypt both your state files and your plan files. In this post, we will explore why state encryption is important and how you can implement it.

Why should you encrypt your state file?

Encrypting your state and plan files is critical in infrastructure management because they often contain sensitive information such as credentials, access keys, and configurations. If these are exposed, your infrastructure can have serious problems, which could lead to severe security breaches. 

Encrypting these files ensures that only authorized users have access to read the information, significantly reducing the risk associated with breaches. Encryption also helps with data integrity because any unauthorized modification of encrypted data would fail to decrypt, alerting you to data tampering. 

Encrypting these files will also help with compliance checks and audits because it shows you adhere to security best practices. This helps prevent legal issues and demonstrates a proactive approach to data security.

How does OpenTofu state encryption work?

Introduced in version 1.7.0, OpenTofu state encryption works through robust encryption methods and key providers. It currently supports the following key providers:  PBKDF2, AWS KMS, GCP KMS, and OpenBao in beta. The encryption method can be either AES-GCM or unencrypted (used only for explicit migration to and from encryption). AES-GCM ensures data integrity by making any unauthorized changes detectable, making your IaC secure and reliable.

OpenTofu uses these mechanisms to encrypt your state data at rest. If you enable it, you won't be able to recover the state/plan files without the appropriate encryption key.

How to configure state encryption?

OpenTofu state encryption is configured through a special "encryption" block that will be part of the "terraform" block.

In the encryption block, you can configure multiple blocks:

  • key_provider - Specifies the key provider for the encryption, can be PBKDF2, AWS KMS, GCP KMS, orOpenBao. Depending on which key provider you select, you will have different configuration options.
  • method - The encryption method to be used, currently the option is AES-GCM which permits 16, 24, and 32-byte keys.
  • state and/or plan - Here you specify the encryption method and a fallback option because OpenTofu lets you automatically roll over your encryption configuration to an old one by having this fallback option.
  • remote_state_data_source - You have the option to also configure encryption for remote state that you leverage through the "terraform_remote_state" datasource.

Configure state encryption through PBKDF2

We will now explore how to use the PBKDF2 key provider to enable state encryption.

terraform {
 encryption {
   key_provider "pbkdf2" "migration_key" {
     passphrase    = "super-passphrase-hard-to-find"
     key_length    = 32
     salt_length   = 16
     hash_function = "sha256"
   }
   method "aes_gcm" "secure_method" {
     keys = key_provider.pbkdf2.migration_key
   }
   state {
     method = method.aes_gcm.secure_method
   }
 }
}

resource "random_pet" "one" {}
Enter fullscreen mode Exit fullscreen mode

Let's run tofu init and tofu apply and see what happens with our state:

tofu init

Initializing the backend...

-----

OpenTofu has created a lock file .terraform.lock.hcl to record the provider
selections it made above. Include this file in your version control repository
so that OpenTofu can guarantee to make the same selections by default when
you run "tofu init" in the future.

OpenTofu has been successfully initialized!

You may now begin working with OpenTofu. Try running "tofu plan" to see
any changes that are required for your infrastructure. All OpenTofu commands
should now work.

If you ever set or change modules or backend configuration for OpenTofu,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
Enter fullscreen mode Exit fullscreen mode 
tofu apply

Plan: 1 to add, 0 to change, 0 to destroy.

Do you want to perform these actions?
 OpenTofu will perform the actions described above.
 Only 'yes' will be accepted to approve.

 Enter a value: yes

random_pet.one: Creating...
random_pet.one: Creation complete after 0s [id=glad-bat]

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.
Enter fullscreen mode Exit fullscreen mode

Now, if we check our state, we won't see any information related to the resources in our state:

{
   "serial": 1,
   "lineage": "92dada5a-e8a8-b917-73b0-25a9a8e19714",
   "meta": {
       "key_provider.pbkdf2.migration_key": "..."
   },
   "encrypted_data": "RL...",
   "encryption_version": "v0"
}
Enter fullscreen mode Exit fullscreen mode

The key itself is derived from the passphrase, key length, number of iterations, salt length, the salt itself (randomly generated on each encryption, and stored in the state file for later decryption), and hash functions each time it's needed. The key won't be stored anywhere on the disk.

Now if we run a tofu state list, we will see the resources in our state file:

tofu state list
random_pet.one
Enter fullscreen mode Exit fullscreen mode

Let's change the passphrase and try to run the command again:

tofu state list
Failed to load state: decryption failed for all provided methods: attempted decryption failed for state: decryption failed: cipher: message authentication failed
Enter fullscreen mode Exit fullscreen mode

You can also try to do other operations such as a tofu plan/apply:

tofu plan
╷
│ Error: Error acquiring the state lock
│
│ Error message: decryption failed for all provided methods: attempted decryption failed for state: decryption failed: cipher: message
│ authentication failed
│
│ OpenTofu acquires a state lock to protect the state from being written
│ by multiple users at the same time. Please resolve the issue above and try
│ again. For most commands, you can disable locking with the "-lock=false"
│ flag, but this is not recommended.
Enter fullscreen mode Exit fullscreen mode

As you can see, because we don't have all the correct information related to encryption, we cannot do any other operations.

Configure state encryption through AWS KMS

The AWS KMS key provider will use the following attributes:

  • kms_key_id - the id of the AWS KMS key you want to use for encryption
  • key_spec - encryption method (e.g. AES_256)
  • region - the region in which the AWS KMS resides 
terraform {
 encryption {
   key_provider "aws_kms" "kms_key" {
     kms_key_id = "18370188-.."
     key_spec   = "AES_256"
     region     = "eu-west-1"
   }
   method "aes_gcm" "secure_method" {
     keys = key_provider.aws_kms.kms_key
   }
   state {
     method = method.aes_gcm.secure_method
   }
 }
}

resource "random_pet" "one" {}
Enter fullscreen mode Exit fullscreen mode

Repeat the init/apply steps, and then check the state file:

{
   "serial": 1,
   "lineage": "d97396a4-7d24-aa29-add7-22abf0a563d8",
   "meta": {
       "key_provider.aws_kms.kms_key": ".."
   },
   "encrypted_data": "...,
   "encryption_version": "v0"
}
Enter fullscreen mode Exit fullscreen mode

As you can see, in the meta block, the key_provider shows a different encryption key.

💡 You might also like:

How do you migrate from an unencrypted state?

Let's take a look at a simple example that generates a random password that we will further use for our database:

resource "random_password" "my_password" {
 length = 8
}

Plan: 1 to add, 0 to change, 0 to destroy.

Do you want to perform these actions?
 OpenTofu will perform the actions described above.
 Only 'yes' will be accepted to approve.

 Enter a value: yes

random_password.my_password: Creating...
random_password.my_password: Creation complete after 0s [id=none]

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.
Enter fullscreen mode Exit fullscreen mode

Now that this password is created, we can look into the state and see the password result clearly:

cat terraform.tfstate | grep \"result\"
           "result": "3_1Wzi5D"
Enter fullscreen mode Exit fullscreen mode

This means that if an attacker intercepts our state file, they can access our sensitive value.

Let's migrate this state file to an encrypted one.

A best practice whenever you are doing any kind of state migration is to back up the current state, so make sure you do that.

Next, we will build our terraform encryption block in the same way as before, but we will use a couple of other parameters to enable the migration. Let's use the AWS KMS key provider as the one we will migrate to:

terraform {
 encryption {
   method "unencrypted" "migrate" {}

   key_provider "aws_kms" "kms_key" {
     kms_key_id = "18370188-..."
     key_spec   = "AES_256"
     region     = "eu-west-1"
   }
   method "aes_gcm" "secure_method" {
     keys = key_provider.aws_kms.kms_key
   }
   state {
     method = method.aes_gcm.secure_method
     fallback {
       method = method.unencrypted.migrate
     }
   }
 }
}
Enter fullscreen mode Exit fullscreen mode

In the first part of the configuration, we've added the unencrypted method. OpenTofu won't otherwise read the state because it could've been manipulated, so we are using this method to tell OpenTofu that the state file was unencrypted initially. We also specify the fallback, which we don't do with a basic AWS KMS encryption. The fallback block is used to handle changes to your key providers.

You can also migrate from one key provider to another. The only difference is that you won't need to use the unencrypted block.

Let's see this in action by running init and apply:

tofu init
---
╷
│ Warning: Unencrypted method configured
│
│ Method unencrypted is present in configuration. This is a security risk and should only be enabled during migrations.
╵

OpenTofu has been successfully initialized!
---
Enter fullscreen mode Exit fullscreen mode 
tofu apply
random_password.my_password: Refreshing state... [id=none]

No changes. Your infrastructure matches the configuration.

OpenTofu has compared your real infrastructure against your configuration and found no differences, so no changes are needed.

Apply complete! Resources: 0 added, 0 changed, 0 destroyed
Enter fullscreen mode Exit fullscreen mode

As you can see, your resources don't change, which is the expected behavior. Now we can check the state file:

{
   "serial": 9,
   "lineage": "d57e7365-0767-179d-1a3c-bbbcb932e245",
   "meta": {
       "key_provider.aws_kms.kms_key": "..."
   },
   "encrypted_data": "...,
   "encryption_version": "v0"
}
Enter fullscreen mode Exit fullscreen mode

Now that we have successfully migrated to the encrypted state, we can remove the references to unencrypted in our encryption block:

terraform {
 encryption {
   key_provider "aws_kms" "kms_key" {
     kms_key_id = "18370188-..."
     key_spec   = "AES_256"
     region     = "eu-west-1"
   }
   method "aes_gcm" "secure_method" {
     keys = key_provider.aws_kms.kms_key
   }
   state {
     method = method.aes_gcm.secure_method
   }
 }
Enter fullscreen mode Exit fullscreen mode

State encryption with Spacelift

Enabling state encryption with Spacelift works out of the box. You just configure it as you would do normally.

We can create a stack based on the AWS KMS example and attach a cloud integration to it to read the KMS key.

encrypt opentofu state

Next, let's apply the code:

Next, let's try to use this state locally without adding the corresponding encryption block. First, we will log in to our Spacelift account from our terminal:

tofu login spacelift.io
...
OpenTofu will request an API token for spacelift.io using OAuth.

Do you want to proceed?
 Only 'yes' will be accepted to confirm.

 Enter a value: yes

OpenTofu must now open a web browser to the login page for spacelift.io.

...
OpenTofu will now wait for the host to signal that login was successful.

---------------------------------------------------------------------------------

Success! OpenTofu has obtained and saved an API token.

The new API token will be used for any future OpenTofu command that must make
authenticated requests to spacelift.io.
Enter fullscreen mode Exit fullscreen mode

Next, I will write the corresponding terraform block for my account and stack:

terraform {
 backend "remote" {
   hostname     = "spacelift.io"
   organization = "saturnhead"

   workspaces {
     name = "tofu_state_encryption"
   }
 }
}
Enter fullscreen mode Exit fullscreen mode

Now we can try to run tofu init:

tofu init

Initializing the backend...

Successfully configured the backend "remote"! OpenTofu will automatically
use this backend unless the backend configuration changes.
Error refreshing state: Unsupported state file format: This state file is encrypted and can not be read without an encryption configuration
Enter fullscreen mode Exit fullscreen mode

As you can see, this results in an error because we don't have the encryption block. Let's add it:

terraform {
 backend "remote" {
   hostname     = "spacelift.io"
   organization = "saturnhead"

   workspaces {
     name = "tofu_state_encryption"
   }
 }
 encryption {
   key_provider "aws_kms" "kms_key" {
     kms_key_id = "18370188-.."
     key_spec   = "AES_256"
     region     = "eu-west-1"
   }
   method "aes_gcm" "secure_method" {
     keys = key_provider.aws_kms.kms_key
   }
   state {
     method = method.aes_gcm.secure_method
   }
 }
}
Enter fullscreen mode Exit fullscreen mode

Now, if we retry the init command we see that we can perform operations against the state.

tofu init
OpenTofu has been successfully initialized!
Enter fullscreen mode Exit fullscreen mode

Key points

Encryption is vital in protecting your data against breaches, and OpenTofu can encrypt both your state and plan files. This ensures that your data remains secure against unauthorized access, and your IaC maintains its integrity and confidentiality.

Written by Flavius Dinu.

Top comments (1)

Read next

siddhantkcode profile image

Querying S3 logs in a flash with DuckDB 🦆

Siddhant Khare -

code42cate profile image

Day 7: Docker Volumes

Jonas Scholz -

i_am_vesh profile image

Getting Started with Terraform- Day 1

Avesh -

lalith_charan profile image

Understanding the Linux Filesystem: An In-Depth Guide for DevOps Engineers

lalith_charan -