In this blog post, we’ll demonstrate how to leverage GitHub Actions (GHA) and Google Workload Identity Federation (WIF) to securely authenticate and create resources on Google Cloud Platform (GCP) using Terraform.
We’ll use two GitHub repositories and two GCloud projects:
Main GitHub Repository:
Contains the Terraform code defining the infrastructure to be provisioned in the Target Project.
Triggers the GitHub Actions workflow upon code changes.
Workflow Repository:
Hosts the GitHub Actions workflow that automates the deployment process.
Authenticates to the Host Project using Workload Identity Federation (WIF).
Executes Terraform commands to apply the infrastructure changes to the Target Project.
Host Google Cloud Project:
Serves as the landing zone for the GitHub Actions workflow.
Configured with WIF, utilises a service account that’s been granted the necessary permissions to access the Target Project.
Does not directly host any infrastructure resources.
Target Google Cloud Project:
Receives the infrastructure changes from the workflow.
Hosts the provisioned resources defined in the Terraform code.
Let’s dive into the setup!
Step 1: Setting Up Google Workload Identity Federation
1.1 - On your host project create a Google Cloud Service Account
Navigate to the Google Cloud Console.
Go to IAM & Admin > Service Accounts.
Create a new service account with the necessary roles for managing your GCP resources.
1.2 - Configure Workload Identity Federation
Go to IAM & Admin > Workload Identity Federation.
Create a Workload Identity Pool and a Provider linked to your GitHub repository.
Follow Google’s official WIF setup guide for detailed instructions. More in a separate Blog Post soon.
Step 2: Permissions
2.1 - Ensure that the target gcp project has a terraform state bucket (ex: project_id-tfstate)
2.2 - Assign relevant roles(roles/editor &
roles/storage.admin) to the Service Account of the Host Project to be able to create/edit resources on the Target Google Project
gcloud projects add-iam-policy-binding gcp_project_name \
--member="serviceAccount:service-acct-name@host_gcp_project.iam.gserviceaccount.com" \
--role="roles/editor"
gcloud projects add-iam-policy-binding gcp_project_name \
--member="serviceAccount:service-acct-name@host_gcp_project.iam.gserviceaccount.com" \
--role="roles/storage.admin"
Step 3: Configure the Main Repo
This repository (e.g., org_name/google_cloud) will contain your Terraform code to manage GCP resources.
Example main.tf:
provider "google" {
project = var.project
region = var.region
}
resource "google_storage_bucket" "bucket" {
name = "${var.project}-bucket"
location = var.region
}
Step 4: Add a Workflow to the Main Repo
Create a GitHub Actions workflow (e.g., .github/workflows/dispatch.yml) in the main repository. This workflow is triggered by events such as push, pull request, or tag creation. When triggered, it makes a GitHub API call to initiate a workflow in another repository (referred to as the Workflow Repo).
Step 5: Configure the Workflow Repo
In the Workflow Repository (Workflow Repo), set up a GitHub Actions workflow dedicated to running Terraform-specific tasks, such as generating a Terraform plan and applying it to the infrastructure. For example, you can name this file .github/workflows/terraform_plan_apply_gcp.yml. Let’s refer to it as the Terraform Workflow.
How It Works
1.Triggering Event:
This Terraform Workflow is designed to be triggered by repository_dispatch events. These events are custom webhook events sent by the main repository via a GitHub API call. When a repository_dispatch event occurs with a specific type, it starts this workflow.
2.Event Types:
You can define specific event types that the workflow should listen to. In this example, the workflow listens for two custom event types:
• terraform_plan: Used to trigger the workflow for generating a Terraform plan.
• terraform_apply: Used to trigger the workflow for applying the Terraform changes.
3.Workflow Configuration:
Add the following YAML configuration to define the trigger mechanism in the Terraform Workflow:
on:
repository_dispatch:
types: [terraform_plan, terraform_apply]
• The on: repository_dispatch block specifies that this workflow listens for repository_dispatch events.
• The types field restricts the workflow to respond only to the specified event types (terraform_plan and terraform_apply).
4.Integration with Main Repository:
In the main repository, another workflow (e.g., .github/workflows/dispatch.yml) sends these repository_dispatch events using a GitHub API call. The event payload includes the event type (e.g., terraform_plan) and any additional data required by the Terraform Workflow.
5.Example Workflow Behavior:
• If the main repository triggers a repository_dispatch event with the type terraform_plan, the Terraform Workflow starts and runs tasks related to generating a Terraform plan.
• Similarly, if the event type is terraform_apply, the workflow executes the Terraform apply process to update the infrastructure.
This mechanism decouples the triggering mechanism in the main repository from the Terraform-specific operations in the Workflow Repo, enabling modular and maintainable workflows.
Scenarios to demonstrate how the Main Repo Workflow Operates
Scenario 1: Push with a Tag
A developer pushes Terraform code to a feature branch and adds a tag (e.g., gcp_project_TFPLAN_01).
The dispatch.yml workflow triggers when a tag matching a specific pattern is pushed (e.g., '[a-z]+-[a-z]+TFPLAN[0-9]+').
The workflow determines the Terraform action (e.g., plan) and triggers the Workflow Repo's Terraform workflow via a GitHub API repository_dispatch event.
Scenario 2: Pull Request Creation
When a pull request is opened from a feature branch to develop, the workflow sends a repository_dispatch event with details such as:
event_type: terraform_plan
GCP project name & PR metadata (e.g., PR number, status=opened, merged=false).
Scenario 3: Pull Request Merge
When a pull request is merged, the workflow sends a repository_dispatch event with:
event_type: terraform_apply
GCP project name & PR metadata (e.g., PR number, status=closed, merged=true).
Full workflow
# This GitHub Actions workflow is designed to trigger a Terraform workflow based on specific events.
name: Trigger Terraform Workflow
on:
push:
tags:
- '[a-z]+-[a-z]+_PLAN_[0-9]+'
# branches:
# - 'feature/*'
pull_request:
types: [opened, synchronize, closed]
branches:
- develop
jobs:
trigger:
runs-on: ubuntu-latest
if: >
github.event_name == 'push' ||
(github.event_name == 'pull_request' &&
(github.event.action == 'opened' ||
github.event.action == 'synchronize' ||
(github.event.action == 'closed' && github.event.pull_request.merged == true)))
steps:
- name: Checkout repository
uses: actions/checkout@v2
# Debug step to print the GITHUB_REF
- name: Retrieve GitHub Data
if: github.event_name == 'push' || github.event_name == 'pull_request'
run: |
echo "GITHUB_REF=${GITHUB_REF}"
TAG_NAME=${GITHUB_REF#refs/tags/}
echo "TAG_NAME=$TAG_NAME" >> $GITHUB_ENV
- name: Print PR Information
if: github.event_name == 'pull_request'
run: |
# Get the latest TAG Name from the source branch
git fetch --tags
# Get latest tag
TAG_NAME=$(git tag --sort=-creatordate | head -n 1)
# Output the tag
if [ -z "$TAG_NAME" ]; then
echo "No tags found on the source branch: $SOURCE_BRANCH"
else
echo "The latest tag on the source branch ($SOURCE_BRANCH) is: $TAG_NAME"
echo "TAG_NAME=$TAG_NAME" >> $GITHUB_ENV
fi
echo "PR Number: ${{ github.event.pull_request.number }}"
echo "PR Action: ${{ github.event.action }}"
echo "PR Merged: ${{ github.event.pull_request.merged }}"
- name: Extract Information from Tag or PR
id: extract_info
run: |
GCP_PROJECT=$(echo $TAG_NAME | cut -d'_' -f1)
echo "GCP_PROJECT=$GCP_PROJECT" >> $GITHUB_ENV
echo "The Tag Name is: $TAG_NAME"
echo "The Target GCP Project is: $GCP_PROJECT"
if [[ "${{ github.event_name }}" == "push" ]]; then
ACTION="plan"
elif [[ "${{ github.event_name }}" == "pull_request" ]]; then
if [[ "${{ github.event.action }}" == "closed" && "${{ github.event.pull_request.merged }}" == "true" ]]; then
ACTION="apply"
else
ACTION="plan"
fi
fi
echo "ACTION=$ACTION" >> $GITHUB_ENV
echo "The Terraform Action is: $ACTION"
- name: Trigger Terraform Workflow for Push Commits
if: github.event_name == 'push'
run: |
echo "This was triggered as a result of the Event: ${{ github.event_name }} commits"
curl -X POST \
-H "Accept: application/vnd.github.v3+json" \
-H "Authorization: Bearer ${{ secrets.GH_DISPATCH_PAT }}" \
https://api.github.com/repos/${{ github.repository }}/dispatches \
-d "{\"event_type\":\"terraform_${{ env.ACTION }}\", \"client_payload\": {\"repository\": \"${{ github.repository }}\", \"project_name\": \"${{ env.GCP_PROJECT }}\", \"tag_name\": \"${{ env.TAG_NAME }}\"}}"
- name: Trigger Terraform Workflow for PR
if: github.event_name == 'pull_request'
run: |
echo "This was triggered as a result of PR Number: ${{ github.event.pull_request.number }} being ${{ github.event.action }}"
curl -X POST \
-H "Accept: application/vnd.github.v3+json" \
-H "Authorization: Bearer ${{ secrets.GH_DISPATCH_PAT }}" \
https://api.github.com/repos/${{ github.repository }}/dispatches \
-d "{\"event_type\":\"terraform_${{ env.ACTION }}\", \"client_payload\": {\"repository\": \"${{ github.repository }}\", \"pr_number\": \"${{ github.event.pull_request.number }}\", \"pr_event\": \"${{ github.event.action }}\", \"pr_merged\": \"${{ github.event.pull_request.merged }}\", \"project_name\": \"${{ env.GCP_PROJECT }}\", \"action\": \"${{ env.ACTION }}\"}}"
How the Terraform Workflow Operates
This workflow is hosted in the Workflow Repo and is triggered by repository dispatch events. Below, we break down each step with detailed explanations and corresponding code.
Step 1: Define Workflow Triggers
The workflow listens for certain event types:
repository_dispatch: Triggered by the Main Repo for Terraform plan and apply actions. More on repository dispatch can be found on the Official GitHub Documentation.
name: Terraform CI/CD
on:
workflow_dispatch:
repository_dispatch:
types: [terraform_plan, terraform_apply]
Step 2: Set Up the Terraform Job
Define a Terraform job that runs on ubuntu-latest with relevant permissions for GitHub Actions to interact with the repository and pull requests.
jobs:
terraform:
runs-on: ubuntu-latest
permissions:
contents: read
id-token: write
pull-requests: write
repository-projects: write
Step 3: Set Environment Variables
Capture the payload data from the triggering event(the main workflow) and set them as environment variables for subsequent steps.
- name: Set ENV variables
run: |
echo "TARGET_GCP_PROJECT=${{ github.event.client_payload.project_name }}" >> $GITHUB_ENV
echo "CLOUDSDK_CORE_PROJECT=${{ github.event.client_payload.project_name }}" >> $GITHUB_ENV
echo "GH_REPOSITORY=${{ github.event.client_payload.repository }}" >> $GITHUB_ENV
GH_REPO_NAME=$(echo "${{ github.event.client_payload.repository }}" | cut -d'/' -f2)
echo "GH_REPO_NAME=${GH_REPO_NAME}" >> $GITHUB_ENV
echo "GH_PR_NUMBER=${{ github.event.client_payload.pr_number }}" >> $GITHUB_ENV
echo "GH_PR_EVENT=${{ github.event.client_payload.pr_event }}" >> $GITHUB_ENV
echo "GH_PR_MERGED=${{ github.event.client_payload.pr_merged }}" >> $GITHUB_ENV
Step 4: Checkout the Code
Clone the Main Repo containing the Terraform code.
- name: Checkout code
uses: actions/checkout@v2
with:
repository: ${{ env.GH_REPOSITORY }}
token: ${{ secrets.GITHUB_TOKEN }}
Step 5: Set Up Terraform
Set up Terraform to run commands such as init, plan, and apply.
- name: Set up Terraform
uses: hashicorp/setup-terraform@v3
Step 6: Authenticate to Google Cloud
Use Workload Identity Federation to securely authenticate with Google Cloud.
- name: Authenticate to Google Cloud
id: authenticate
uses: google-github-actions/auth@v2
with:
create_credentials_file: true
workload_identity_provider: ${{ secrets.GCP_WORKLOAD_IDENTITY_PROVIDER }}
service_account: ${{ secrets.GCP_SERVICE_ACCOUNT }}
Step 7: Initialize Terraform
Set the GCP project and initialize Terraform with the remote backend configuration.
- name: Terraform Init
id: init
run: terraform init -backend-config="bucket=$TARGET_GCP_PROJECT-tfstate"
working-directory: ${{ env.TF_WORKING_DIR }}
Step 8: Validate Terraform Code
Ensure the Terraform configuration is correctly formatted and syntactically valid.
- name: Terraform Format
id: fmt
run: terraform fmt
working-directory: ${{ env.TF_WORKING_DIR }}
- name: Terraform Validate
id: validate
run: terraform validate
working-directory: ${{ env.TF_WORKING_DIR }}
Step 9: Generate Terraform Plan
Generate a plan and output the details for review.
- name: Terraform Plan
id: plan
run: terraform plan -var-file="$TARGET_GCP_PROJECT.tfvars" -out=tfplan
working-directory: ${{ env.TF_WORKING_DIR }}
- run: terraform show -no-color tfplan
id: show
working-directory: ${{ env.TF_WORKING_DIR }}
## We will use the output of terraform show to write the plan as a comment to the pull request
Step 10: Comment on Pull Requests
If triggered by a pull request, post the plan as a comment for review.
- name: PR Comment
uses: actions/github-script@v7
if: github.event.action == 'terraform_plan' && ( env.GH_PR_EVENT == 'opened' || env.GH_PR_EVENT == 'synchronize' )
env:
PLAN: "terraform\n${{ steps.show.outputs.stdout }}"
with:
github-token: ${{ secrets.GH_PAT }}
script: |
const { data: comments } = await github.rest.issues.listComments({
owner: context.repo.owner,
repo: process.env.GH_REPO_NAME,
issue_number: process.env.GH_PR_NUMBER,
})
const botComment = comments.find(comment => comment.body.includes('Terraform Format and Style'))
const output = `#### Terraform Plan\n\`\`\`\n${process.env.PLAN}\n\`\`\``
if (botComment) {
github.rest.issues.updateComment({
owner: context.repo.owner,
repo: process.env.GH_REPO_NAME,
comment_id: botComment.id,
body: output
})
} else {
github.rest.issues.createComment({
owner: context.repo.owner,
repo: process.env.GH_REPO_NAME,
issue_number: process.env.GH_PR_NUMBER,
body: output
})
}
Step 11: Apply the Terraform Plan
If the event is terraform_apply, apply the plan to create resources.
- name: Terraform Apply
if: github.event.action == 'terraform_apply'
id: apply
run: terraform apply -auto-approve tfplan
working-directory: ${{ env.TF_WORKING_DIR }}
By integrating GitHub Actions and Google Workload Identity Federation, you can establish a secure, automated CI/CD pipeline for managing GCP resources using Terraform. This approach ensures that Terraform plans are reviewed, validated, and applied only after thorough approval, enhancing both security and operational efficiency.
Top comments (1)
This is such a practical tutorial for streamlining GCP resource management with GitHub Actions and WIF! The integration of Terraform adds so much flexibility. For those exploring managed cloud hosting, platforms like Cloudways simplify many hosting tasks for PHP apps. Combining Cloudways with GCP for hybrid solutions could be an interesting approach too!