DEV Community

Cover image for GitHub Actions - Azure Terraform CI/CD
Cole Heard
Cole Heard

Posted on

GitHub Actions - Azure Terraform CI/CD

#azure #devops #terraform #githubactions

Terraform is my preferred tool for Azure resource creation. I still run some Terraform commands from my local shell, but I've made a real effort to execute the bulk of my changes with GitHub Actions.

This post will detail my basic Azure Terraform pipeline. If you're looking for more advanced content, this is not the article for you.

First, I will highlight some workflow prerequisites. Once those have been covered, I will walkthrough the pipeline, step-by-step.

Take a seat and let's get started.

Homer - Take a Seat

Prerequisites

The workflow does not exist in a vacuum - there are a few things that need to be configured outside of the workflow itself.

Branch protection: Enforced Branch protection requires the use of pull requests and merges. The code will always be merged from another branch to main, the pull request will detail the specific changes to be made, and the main branch will more reliably reflect the current state of the environment.

GitHub Token: A fine-grained token grants access to private repositories. The token is stored as a secret for later use.

Azure Credentials: An app registration is used to authenticate the runner to Azure. The app registration's associated client secret - along with the subscription, tenant, and management endpoint are stored as a GitHub secret (in JSON syntax). 



{                                   
    "clientId": "12345678-1234-5678-9012-345678901234",
    "clientSecret": "0000000000000000000000000000000000000000",
    "subscriptionId": "1010101-0101-0101-0101-010101010101",
    "tenantId": "ABCDEFG-HIJK-LMNO-PQRS-123456789012",
    "managementEndpointUrl": "https://management.core.windows.net/"
}
Enter fullscreen mode Exit fullscreen mode

Azure Storage Account: This is an Azure focused project, so an azurerm backend seemed appropriate. An Azure Storage Account was created to store Terraform's statefile. The app registration's service principal has contributor rights to the storage account - Terraform will authenticate with the same secret stored above (more on that later).

The Workflow

Now that the prerequisites have been addressed, we will dissect the pipeline, tfpipeline.yaml.

Getting Started

The workflow will only begin once a trigger condition has been met, as described by on:



name: "Basic Terraform Pipeline"

on:
  pull_request:
  push:
    branches:
      - main

jobs:
  terraform:
    name: 'Terraform - Ubuntu'
    runs-on:    Self-hosted

    defaults:
      run:
        shell: bash
Enter fullscreen mode Exit fullscreen mode

This workflow is waiting for one of two events to occur:

  1. A pull request is opened.
  2. A push is made to the main branch.

On these conditions, the self-hosted runner starts the Job as defined below.

The checkout step clones the repository down to the runner.



    steps:
      - name: Code Checkout
        id: checkout
        uses: actions/checkout@v3
Enter fullscreen mode Exit fullscreen mode

Azure Authentication

The Azure Login Action authenticates with the Azure JSON secret.



      - name: Azure Authentication
        id: login
        uses: azure/login@v1
        with:
          creds: ${{ secrets.AZJSON }}
Enter fullscreen mode Exit fullscreen mode

The parse step uses jq to parse the Azure JSON. The key values are echoed to environmental variables for use by Terraform.

The variables are ultimately passed to $GITHUB_ENV, one of GitHub Actions default environmental variables. 



      - name: JSON Parse
        id: parse
        env:
          AZJSON: ${{ secrets.AZJSON }}
        run: |
          ARM_CLIENT_ID=$(echo $AZJSON | jq -r '.["clientId"]')
          ARM_CLIENT_SECRET=$(echo $AZJSON | jq -r '.["clientSecret"]')
          ARM_TENANT_ID=$(echo $AZJSON | jq -r '.["tenantId"]')
          ARM_SUBSCRIPTION_ID=$(echo $AZJSON | jq -r '.["subscriptionId"]')
          echo ARM_CLIENT_ID=$ARM_CLIENT_ID >> $GITHUB_ENV
          echo ARM_CLIENT_SECRET=$ARM_CLIENT_SECRET >> $GITHUB_ENV
          echo ARM_TENANT_ID=$ARM_TENANT_ID >> $GITHUB_ENV
          echo ARM_SUBSCRIPTION_ID=$ARM_SUBSCRIPTION_ID >> $GITHUB_ENV
Enter fullscreen mode Exit fullscreen mode

Access to Private Repositories

The GitHub token is streamed to .netrc. Git is then configured to use https for the logon. These git changes provide access to the private registry configured within the same GitHub org.

This step also disables git's detached head warnings.



      - name: GitHub Token
        id: token
        env:
          TOKEN: ${{ secrets.GHTOKEN }}
        run: |
          echo "machine github.com login x password ${TOKEN}" > ~/.netrc
          git config --global url."https://github.com/".insteadOf "git://github.com/"
          git config --global advice.detachedHead false
Enter fullscreen mode Exit fullscreen mode

Terraform Prep

Terraform is installed and initialized.

The azurerm backend is configured to use environmental variables created during the parse step. Only a single secret is maintained for all Azure authentication. 



      - name: Install Terraform
        uses: hashicorp/setup-terraform@v2.0.3
        with:
          terraform_version: 1.3.5

      - name: Terraform Init
        id: init
        run: |
          terraform init
Enter fullscreen mode Exit fullscreen mode

Checkov

Checkov is an open-source static code analysis tool.
The tool compares the code to defined policy - the policies can be out-of-the-box security checks or custom .py or .yaml files.

Here Pip installs Checkov.



      - name: Install Checkov
        id: checkov
        if: github.event_name == 'pull_request'
        run: |
          pip install checkov
Enter fullscreen mode Exit fullscreen mode

Notice the if: expression. These steps only run if the trigger event was a pull request.

Checkov will now run. 



      - name: Checkov Static Test
        id: static
        if: github.event_name == 'pull_request'
        run: |
          checkov -d . --download-external-modules true
Enter fullscreen mode Exit fullscreen mode

Checkov should run after Terraform init; any modules called by Terraform are installed during init and we'll want Checkov to test their code as well.

CheckovOutput

More Terraform

The next few steps are Terraform staples. We run format, validate, and plan.



      - name: Terraform Format
        id: fmt
        run: terraform fmt -check -recursive
        continue-on-error: true

      - name: Terraform Validate
        id: validate
        run: terraform validate -no-color     

      - name: Terraform Plan
        id: tplan
        env: 
          TF_VAR_secret: '${{ secrets.example_secret }}'
        run: |
            terraform plan -no-color
Enter fullscreen mode Exit fullscreen mode

Checking the Plan

This step requires another Terraform plan run. The previous plan did not output to a file - the console output from tplan will be used later. 



      - name: Checkov Plan Test
        id: cplan
        if: github.event_name == 'pull_request'
        env: 
           TF_VAR_secret: '${{ secrets.example_secret }}'
        run: |
            terraform plan --out plan.tfplan
            terraform show -json plan.tfplan > tfplan.json
            ls
            checkov -f tfplan.json --framework terraform_plan
Enter fullscreen mode Exit fullscreen mode

Pull Request Comment

This step uses the GitHub Script action.

A comment will be created on the pull request. The outcome of many previous steps is displayed for review. Additionally, the full output of the Terraform plan is available as well.

Much of this step's code was borrowed from the Setup Terraform Action documentation.



      - name: Pull Request Comment
        id: comment
        uses: actions/github-script@v3
        if: github.event_name == 'pull_request'
        env:
          TPLAN: "terraform\n${{ steps.tplan.outputs.stdout }}"
        with:
          github-token: ${{ secrets.GHTOKEN }}
          script: |
            const output = `
            ### Pull Request Information
            Please review this pull request. Merging the PR will run Terraform Apply with the plan detailed below.

            #### Terraform Checks
            Init: \`${{ steps.init.outcome }}\`
            Format: \`${{ steps.fmt.outcome }}\`
            Validation: \`${{ steps.validate.outcome }}\`
            Plan: \`${{ steps.tplan.outcome }}\`

            #### Checkov
            Static: \`${{ steps.static.outcome }}\`
            Plan: \`${{ steps.cplan.outcome }}\`

            <details><summary>Plan File</summary>

            \`\`\`${process.env.TPLAN}\`\`\`

            </details>

            `
            github.issues.createComment({
              issue_number: context.issue.number,
              owner: context.repo.owner,
              repo: context.repo.repo,
              body: output
            })
Enter fullscreen mode Exit fullscreen mode

This is what the comment looks like:

PR Comment

If the trigger event was a pull request, the workflow ends here.

Terraform Apply

The workflow only runs Terraform apply when the push occurs on the main branch. 



      - name: Terraform Apply
        id: apply
        if: github.ref == 'refs/heads/main' && github.event_name == 'push'
        env:
          TF_VAR_domain_pass: '${{ secrets.DOMAIN_JOIN_PASS }}'
          TF_VAR_local_pass: '${{ secrets.LOCAL_ADMIN_PASS }}'
          TF_VAR_workspace_key: '${{ secrets.LA_WORKSPACE_KEY }}'
        run: terraform apply -auto-approve
Enter fullscreen mode Exit fullscreen mode

That's all. The workflow tour is finished.

Here is what it looks like all put together:

Overview

Wrapping up

I've found a lot of value deploying resources with this workflow:

  • Resources will be created using the same method, every deployment.
  • With Branch protection enabled, approvals can easily be incorporated into the deployment process.
  • Deploying resources with a purpose-built service principal follows the principal of least privilege.
  • Automated policy checks ensures that they're always being run.

If you enjoyed this article, take a look at my SharePoint Framework pipeline.

Until next time!

Ralph Wiggum - Goodbye

Top comments (1)

Collapse
 
ndrone profile image
Nicholas Drone •

Nice write up. A couple things you shouldn't need the azure authentication step. Terraform alone should be enough. Also Azure and GitHub now allow OIDC from each other, so I would setup a service principal within Azure and then scope it to your GitHub repository. Removing the need for the client secret and adding another layer of security with OIDC.

Read next

teo_nordic_d6988ee4c1221e profile image

Exploring Java's Role in Cloud Computing and AI for 2024

Teo Nordic -

siddhantkcode profile image

When k6 eats up your RAM: Slashing memory usage in load tests

Siddhant Khare -

haamid profile image

Run Gitlab CI jobs on a specific runner

Hamid Haghdoost -

omiossec profile image

Conditional deployment in Azure Bicep

Olivier Miossec -