Rene Hernandez

Posted on Oct 2, 2020 • Originally published at bitsofknowledge.net on Sep 30, 2020

First steps with Terraform in AWS

#terraform #aws #infrastructure #cloud

Terraform is a cloud-agnostic provisioning tool created by Hashicorp. It allows you manage your infrastructure in sane, safe and efficient manner by automating the proviisioning of your cloud resources (server, databases, DNS) in a declarative way, as well as leverage version control systems to keep track of the history of changes.

In this post, we are going to go over how to setup Terraform to work with AWS. If you want to go straight to the code, you can check it out at the setup example

Installing terraform #

There are different ways to install terraform depending on you operating system

Chocolatey on Windows #

choco install terraform

Homebrew on OS X #

Using the new hashicorp tap (recommended)

brew install hashicorp/tap/terraform

Using the community tap

brew install terraform

Linux #

Installing on Linux dependent on the distribution you are running. For Ubuntu/Debian:

Add the HashiCorp gpg key

curl -fsSL https://apt.releases.hashicorp.com/gpg | sudo apt-key add -

Add the HashiCorp repository

sudo apt-add-repository "deb [arch=amd64] https://apt.releases.hashicorp.com $(lsb_release -cs) main"

Update and install

sudo apt-get update && sudo apt-get install terraform

For more information and different ways to install, check the installation pages

Connecting AWS with terraform #

Now that we have the cli installed, let's get started connecting AWS with Terraform to manage the infrastructure.

Prerequisites #

To execute the rest of the project, you'll need to follow the next steps or use your existing configured AWS account.

An AWS account (a dedicated new one to execute terraform preferably, although it is OK to use your own AWS credentials for testing purposes)
The AWS CLI installed.
Configure your local aws cli with a dedicated terraform [profile

aws configure --profile terraform

Using the above command, input at the prompt your AWS Access Key ID and Secret Key

Defining the AWS provider #

According to the Terraform documentation, a provider is essentially a plugin that offers a set of abstraction for certain API resources and their interactions. Usually, each provider focuses on a specific infrastructure platform.

Terraform needs to know which provider to download from the Terraform Registry. For that, we can use the terraform block to list all the providers that the code will use. For our scenario, we can list the aws provider as shown below:

terraform {
  required_providers {
    aws = {
      source = "hashicorp/aws"
      version = "~> 3.0"
    }
  }
  required_version = ">= 0.13"
}

Now that we listed the aws provider, let's add the configuration data for the AWS provider. Below, we are using the us-east-2 region and loading the credentials to connect to AWS from the terraform profile we created on the Prerequisites section.

provider "aws" { 
  region = "us-east-2" 
  profile = "terraform"
}

For alternative means of authentication, dive in the docs and always make sure that you don't store your AWS credentials in plaintext on terraform.

At this point, you are ready to use Terraform to automate your infrastructure. In the next section, we'll dive into storing the backend information remotely to facilitate team workflows.

S3 for remote backend #

First, I'll briefly mention some of the advantages of using a remote backend:

Improves working in a team since state can be protected with locks to avoid multiple interactions at the same time and possible corruptions.
It helps keep sensitive data off disk. Terraform stores the state in plaintext and if using a backend it will only be stored on the backend location (e.g. S3 bucket)
If the backend supports remote operations, it means that terraform apply can be executed on the backend instead of your local machine, for an overall improved experience.

Before going through the code, let's briefly look at what Terraform state entails.

Terraform state #

Terraform stores the state of your infrastructure for several reasons:

Mapping to the Real World: Terraform maps each configuration to a corresponding resource in the real world and it leverages the state to verify that no two configurations elements represent the same endpoint
Metadata: It includes tracking knowledge such as resource dependencies or workflows that are necessary for resources to work as expected.
Performance: Optionally, the state can be considered as a source of truth and therefore API request to the providers are done only when specified. This helps improve performance for large teams.

This state is stored in a file, usually called terraform.tfstate using a JSON format. Let's move on to the implementation

Implementation #

To set up the remote backend, we need the following resources:

A S3 bucket where the terraform state will stored
A DynamoDB table to lock the access to the state file
IAM policies to give permission to the user to access the S3 bucket and the DynamoDB table (required if you are using an IAM user with more restrictive credentials than just AdministratorAccess access)

S3 bucket #

resource "aws_s3_bucket" "terraform_state_storage" {
  bucket = "terraform-remote-state-storage-s3"
  acl = "private"
  tags = {
    name = "Terraform Storage"
    dedicated = "infra"
  }
  versioning {
    enabled = true
  }
  lifecycle {
    prevent_destroy = true
  }
}

The S3 bucket as shown above, gets created with the following settings:

Private Access List Control (ACL) grant to limit access to the bucket.
Versioning enabled to allow for state recovery in case of accidental deletion and file corruption
Prevention of accidental destruction of the S3 bucket while running terraform operations

DynamoDB table #

# create a dynamodb table for locking the state file.
# this is important when sharing the same state file across users
resource "aws_dynamodb_table" "terraform_state_lock" {
  name = "terraform-state-lock"
  hash_key = "LockID"
  read_capacity = 20
  write_capacity = 20
  attribute {
    name = "LockID"
    type = "S"
  }
  tags = {
    name = "DynamoDB Terraform State Lock Table"
    dedicated = "infra"
  }
  lifecycle {
    prevent_destroy = true
  }
}

The DynamoDB table gets configured with the following properties:

A LockID hash key of type string, so that all items created by terraform operations are stored together in the same bucket
The read and write capacity per seconds for the table. This specifies how read/write operations are we allowed to execute against the table
Similar to the S3 bucket above, the resource is created with prevention of accidental destruction while running terraform operations.

IAM policies #

data "aws_iam_policy_document" "terraform_storage_state_access" {
  statement {
    effect = "Allow"
    actions = ["s3:ListBucket"]
    resources = [aws_s3_bucket.terraform_state_storage.arn]
  } 
  statement {
    effect = "Allow"
    actions = ["s3:GetObject", "s3:PutObject"]
    resources = ["${aws_s3_bucket.terraform_state_storage.arn}/terraform.tfstate"]
  }
}

# Creates the IAM policy to allow access to the bucket
resource "aws_iam_policy" "terraform_storage_state_access" {
  name = "terraform_storage_state_access"
  policy = data.aws_iam_policy_document.terraform_storage_state_access.json
}

# Assigns the policy to the terraform user
resource "aws_iam_user_policy_attachment" "terraform_storage_state_attachment" {
  user = "terraform" policy_arn = aws_iam_policy.terraform_storage_state_access.arn
}

The above IAM policy describes the permission for a user to access the S3 bucket:

Ability to list the bucket
Get objects from the bucket
Add new objects to it

As part of the setup, the policy is attached to the terraform user, so it can have access to S3 bucket.

data "aws_iam_policy_document" "dynamodb_access" {
  statement {
    effect = "Allow"
    actions = ["dynamodb:GetItem", "dynamodb:PutItem", "dynamodb:DeleteItem"] 
    resources = ["arn:aws:dynamodb:*:*:table/terraform-state-lock"]
  }
}

# Creates the IAM policy to allow access to the dynamoDB
resource "aws_iam_policy" "dynamodb_access" {
  name = "dynamodb_access"
  policy = data.aws_iam_policy_document.dynamodb_access.json
}

# Assigns the policy to the terraform user
resource "aws_iam_user_policy_attachment" "dynamodb_attachment" {
  user = local.terraform_user policy_arn = aws_iam_policy.dynamodb_access.arn
}

Similar to the previous IAM policy, this IAM policy describes the permission given to the user when accessing the DynamoDB table:

Get items from the table
Add items to the table
Remove items from the table

The policy is then attached to the terraform user, so it can have access to the DynamoDB table.

Remote backend #

Finally, to inform terraform to use the s3 bucket as remote backend, just add the following block:

terraform {
  backend "s3" {
    bucket = "terraform-state-storage"
    key = "terraform.tfstate"
    region = "us-east-2"
    dynamodb_table = "terraform-state-lock"
    profile = "terraform"
  }
}

Resources #

Below, it is a condensed list of all the resources mentioned throughout the posts as well as a few others I consider may be of interest to deepen your knowledge.

Providers:

Terraform remote storage:

S3:

DynamoDB:

IAM:

Conclusions #

To sum up, this post provided an intro to Terraform in AWS. It covered how to install Terraform and configure it to manage AWS resources. As further steps, we went through the steps of configuring S3 and DynamoDB for managing the Terraform state remotely with locking included. This is a desired setup if working on a team. If you want to go straight to the code, go check it out at renehernandez/aws-terraform-examples

Thanks you so much for reading this post. Hope you enjoyed reading it as much as I did writing it. See you soon and stay tuned for more!!

DEV Community