Intro

Being a cloud developer can be hard but there are tons of tools and resources to help smooth the hard edges. One of the tools at our disposal is Terraform, an open-source infrastructure as a code software tool that provides a consistent CLI workflow to manage hundreds of cloud services. But how exactly Terraform is able to connect to those services and create our infrastructure? And can we find the best way to manage our resources? Let's see it.

What you shouldn't do!

Specifying credentials in provider's block

For the love of everything that is good, please don't ever do this if you don't want to quickly find out a new world of grief and pain:

In AWS and Azure documentation, you will find some warning with mild words and a funny little warning block saying this is not recommended; this doesn't even start to convey how much bad practice is hardcoding credentials, so I'll tell you more bluntly:

🔥 DON'T. NEVER. EVER. HARDCODE. CREDENTIALS! 🔥

The internet is full of horror stories of people lazy or careless enough to find themselves with big surprises in their next cloud provider bill. In the most recent one, a guy found out a 65k bill because he "handled source code with hardcoded credentials to other devs".

Do you want to become part of this group? There are better ways to connect your Terraform with your cloud accounts, so let's follow best practices, shall we?

What you should handle with care

Putting credentials in environment variables

The first thing you can do to connect Terraform with your providers is to leverage environment variables. You need to export all the data needed to connect into the default environment variables and you're good to go. Just declare an empty provider and Terraform will automatically pick them up and run.

export ARM_SUBSCRIPTION_ID="00000000-0000-0000-0000-000000000000"
export ARM_TENANT_ID="00000000-0000-0000-0000-000000000000"
export ARM_CLIENT_ID="00000000-0000-0000-0000-000000000000"
export ARM_CLIENT_SECRET="00000000-0000-0000-0000-000000000000"

export AWS_ACCESS_KEY_ID="0000000000000"
export AWS_SECRET_ACCESS_KEY="0000000000000000000000000000000"
export AWS_DEFAULT_REGION="us-east-1"

provider "azurerm" {
  features {}
}

provider "aws" {}

Only downside of this approach is that managing this environment variables manually is a real pain. How cool would be to have an automated way to handling this? ;)

🔥 Refrain from exporting directly in your .bash_rc, .profile or any other automatically sourced configuration files. Those file are not encrypted you it's a very similar situation as the hardcoded credentials. Forewarned is forearmed.

What you should do! (finally)

Here things will split a bit for AWS and Azure, but I'll try to keep them as consistent as possible. We'll mostly look at interactive ways so keep that in mind.

On Azure

Logging through the CLI

This is the default and most secure method for interactively working with Terraform. By running the az login command on the Azure CLI it will export the parameters needed to work into environment variables. The only problem you can incur is by having multiple tenants; in this case, you need to specify in the Terraform Azure provider the one you will use.

provider "azuread" {
  tenant_id = "00000000-0000-1111-1111-111111111111"
}

Using service principals

Service Principal authentication is usually more suitable for automated workflows like for CI/CD or running application scenarios. This type of authentication decouples the authentication process from any specific user login and allows for managed access control.

We can see them as the Azure counterpart of IAM Roles in AWS, and one trivial advantage would be that it can be shared between people without having to resort to groups or tie directly to an identity. In essence, by using a Service Principal, you avoid creating fake users in Azure AD to manage authentication when you need to access resources.

On AWS

Assuming Roles

Assuming a role is the single most simple and secure way to interact with your cloud resources through Terraform. It comes easy as defining the assume_role parameter in the provider and filling out the role you want to assume, the name of the session (it will be handy if you need to trace calls through monitoring systems such as CloudTrail), and an optional external_id that serves as a secret.

provider "aws" {
  assume_role {
    role_arn     = "arn:aws:iam::ACCOUNT_ID:role/ROLE_NAME"
    session_name = "SESSION_NAME"
    external_id  = "EXTERNAL_ID"
  }
}

Overall is a good method but it suffers the same problem as the hardcoded credentials in the sense that when you commit it into your repository anyone can put their hands on this and manage your account.

Shared Credentials file

You can also make all those information external by telling terraform to look up a shared credential file. The default location is $HOME/.aws/credentials on Linux and macOS, or "%USERPROFILE%\.aws\credentials" on Windows. This way the only information you need to exchange with Terraform is the name of the named profile and the CLI will automatically perform the assume role for you.

provider "aws" {
  region                  = "us-west-1"
  shared_credentials_file = "/Users/tf_user/.aws/creds"
  profile                 = "customprofile"
}

This can be further enhanced by using multiple named profiles.

No MFA support, Leapp to the rescue!

Currently, Terraform doesn't support the mfa_serial while assuming roles according to this issue:

Doesn't ask MFA token code when using assume_role with MFA required #2420

jsi-p posted on Nov 23, 2017

When using multiple AWS accounts it's good practice to only allow access via AssumeRole from a master account. This can be done with or without requiring MFA. Terraform supports assume_role with s3 state file and aws provider configurations, but doesn't seem to ask the MFA token code when one is required. This prevents using AssumeRole for credentials when MFA is required.

AWS documentation describing MFA with cross account AssumeRole: http://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_mfa_configure-api-require.html#MFAProtectedAPI-cross-account-delegation

Terraform Version

$ terraform --version
Terraform v0.11.0

Affected Resource(s)

Both of these support assume_role, so they should also support asking for MFA token code:

• S3 backend configuration
• AWS provider configuration

Terraform Configuration Files

terraform {
  backend "s3" {
    bucket = "terraform-state-bucket"
    key = "tf-state"
    region = "eu-west-1"
    role_arn = "arn:aws:iam::916005212345:role/OrganizationAccountAccessRole"
  }
}
provider "aws" {
  region = "eu-west-1"
  assume_role {
    role_arn = "arn:aws:iam::916005212345:role/OrganizationAccountAccessRole"
    session_name = "terraform-session"
  }
}

Actual Behavior (with DEBUG)

$ TF_LOG=debug terraform init
2017/11/23 13:36:12 [INFO] Terraform version: 0.11.0  
2017/11/23 13:36:12 [INFO] Go runtime version: go1.9.2
2017/11/23 13:36:12 [INFO] CLI args: []string{"/usr/local/Cellar/terraform/0.11.0/bin/terraform", "init"}
2017/11/23 13:36:12 [DEBUG] Attempting to open CLI config file: /Users/***/.terraformrc
2017/11/23 13:36:12 [DEBUG] File doesn't exist, but doesn't need to. Ignoring.
2017/11/23 13:36:12 [INFO] CLI command args: []string{"init"}
2017/11/23 13:36:12 [DEBUG] command: loading backend config file: /Users/***

Initializing the backend...
2017/11/23 13:36:12 [WARN] command: backend config change! saved: 16375281725947963338, new: 2383462577283113429
Backend configuration changed!

Terraform has detected that the configuration specified for the backend
has changed. Terraform will now reconfigure for this backend. If you didn't
intend to reconfigure your backend please undo any changes to the "backend"
section in your Terraform configuration.


2017/11/23 13:36:12 [INFO] Building AWS region structure
2017/11/23 13:36:12 [INFO] Building AWS auth structure
2017/11/23 13:36:12 [INFO] Setting AWS metadata API timeout to 100ms
2017/11/23 13:36:13 [INFO] Ignoring AWS metadata API endpoint at default location as it doesn't return any instance-id
2017/11/23 13:36:13 [INFO] Attempting to AssumeRole arn:aws:iam::***:role/OrganizationAccountAccessRole (SessionName: "", ExternalId: "", Policy: "")
2017/11/23 13:36:13 [INFO] AWS Auth provider used: "SharedCredentialsProvider"
2017/11/23 13:36:13 [DEBUG] plugin: waiting for all plugin processes to complete...
Error initializing new backend: 
Error configuring the backend "s3": The role "arn:aws:iam::***:role/OrganizationAccountAccessRole" cannot be assumed.

  There are a number of possible causes of this - the most common are:
    * The credentials used in order to assume the role are invalid
    * The credentials do not have appropriate permission to assume the role
    * The role ARN is not valid

Please update the configuration in your Terraform files to fix this error
then run this command again.

View on GitHub

Leapp can work around this specific scenario by performing the assume role, prompting for the MFA token, and injecting into the credentials file the temporary credentials and tokens to let you use Terraform without further interaction. Nice, isn't it?

Conclusions

As we've seen there are a lot of ways to interact with Terraform for both AWS and Azure. When choosing our access methods we should always think about the security concerns and avoid at all costs manual interactions with the configurations file. For all of this, Leapp is the best tool to achieve both security and automation, abstracting away all underlying interactions and letting cloud developers focus on their tasks.

If you found this interesting, have any questions or you just want to drop a DM and connect you can find me on twitter or join our slack community. See you soon!