Introduction
As many of you will already know, I recently started an exciting new role as Head of Technology at FirstPort, having previously been at a great consultancy called DevOpsGroup.
A key part of my role is delivering FirstPort’s vision of ‘People First’ technology. To do this, it is imperative that I select the right technology to underpin the delivery of services that help make customers’ lives easier.
Today I want to talk about my selection of GitHub Enterprise
GitHub is a best-in-breed, secure, collaborative, code hosting platform with built-in CI/CD capabilities with over 56 million users.
Many organisations use it, and all of them have to manage users, repositories, branch rules, and much more. Standardising all of this is quite challenging and involves creating things like CLI tools, scripts, and other ways to try to automate how people deal with GitHub.
Here at FirstPort, I decided to use Terraform to manage and standardise all of the repository and user management, and today I want to share with you how we are doing this!
Team and user management
If you are using GitHub enterprise, you will probably use Single Sign-on (SSO). This does the job but isn’t great, especially when having to manage collaborators outside of your Identity Provider (However, you can use Terraform to manage sync groups) - This still leaves you to manage repositories, labels, pull requests all manually. In our case, I decided to use Terraform's GitHub provider to automate most of this work.
Firstly, I created the teams:
# Creates a parent team
resource "github_team" "engineering" {
name = "engineering"
description = "This is a parent team"
privacy = "closed"
}
# Creates a sub team of front-end developers
resource "github_team" "frontend" {
name = "frontend"
description = "This is a sub team "
parent_team_id = github_team.engineering.id
privacy = "closed"
}
Every time we have someone new join, anyone at FirstPort can open a PR and invite them to the team.
# Invites a user to the organization
resource "github_membership" "anewstarter" {
username = "firstportuser"
role = "member"
}
# Adds the user to a team
resource "github_team_membership" "frontend-firstportuser1" {
username = github_membership.firstportuser.username
team_id = github_team.frontend.id
role = "maintainer"
}
This will run in a GitHub Actions workflow, and the user will receive an invite to join our organisation.
Repository creation
This is probably the main reason why I decided to use Terraform for managing GitHub. If repositories aren't standardised this can create a chaotic situation if the way engineers open issues, raise PRs etc is different in every repository. To avoid this I decided to create most of our repositories in the same way.
A few things were very important to us that every repository had to have. Let's go through the mains ones:
Release notes
I wanted to provide a way to be able to easily create release notes, to enhance transparency on what we are delivering. I chose to use a tool called Release Drafter in our workflow. I wanted to configure how labels would be used in every repository, and for that, we standardised all labels across all repositories. I use: docs, dependencies, bug, feature, and maintenance.
Every time someone opens a PR, they need to select at least one of these labels to define how this PR is categorised (This is enforced using the Enforce Label Action). When a PR is merged, I use the GitHub Action release drafter, which creates a draft release with the commits from the PR into one of those categories. This helps us to create release notes for all of our services!
This is how the release notes would look like:
Every time someone creates a new repo, the release drafter comes pre-configured and you can start enjoying good release notes without any effort.
Branch protection
Setting up rules for when you can merge a branch is very important for code quality, and in our case for legal reasons (at FirstPort, we deal with personal user data, so all changes must go through strict code quality checks). GitHub gives the possibility for you to configure branch protection rules to solve that.
I wanted to define the same set of basic branch protection for all repositories. This was easily solved with our Terraform module.
Terraform module
To achieve all of this, I created a Terraform module that manages users, teams, and repositories. Here is an example on how we use the module:
module "my-firstport-repo" {
source = "./modules/repo"
version = "~> 1.0"
name = "my-firstport-repo"
description = "A firstport repository description"
visibility = "private"
delete_branch_on_merge = true
vulnerability_alerts = true
teams = { (data.github_team.engineering.id) = "push" }
collaborators = { (data.github_user.external.username) = "push" }
labels = {
"type: bug" = "d73a4a"
"type: docs" = "0f727f"
"type: feature" = "a2eeef"
"type: maintenance" = "a5f7da"
"type: dependencies" = "0366d6"
}
branch_protection = {
master = {
enforce_admins = true
push_restrictions = []
required_status_checks = {
strict = true
contexts = []
}
required_pull_request_reviews = {
dismiss_stale_reviews = true
require_code_owner_reviews = true
dismissal_restrictions = []
}
}
}
}
GitHub Actions
GitHub Actions is our CI tool of choice. GitHub Actions is a hosted runner service provided by GitHub. Any user can write individual tasks, called actions, and put them together into a workflow. These workflows can trigger off numerous events, such as pull requests, comments, labels, releases, and so forth. I think of it as having a box of LEGO bricks that can be put together as needed; I can build a space ship or a pirate ship as my heart desires.
Users are free to write their own actions or consume them from the GitHub Marketplace. For example, the action that performs code checkout is written by GitHub and is on the Marketplace. For a more in-depth introduction to GitHub Actions, I suggest reading the Getting started with GitHub Actions documentation.
For the purpose of this article, I am using GitHub Actions to construct a workflow to provide CI functionality.
There is one workflow that runs on every PR:
name: Terraform Plan
on:
- pull_request
jobs:
terraform:
name: Terraform
runs-on: ubuntu-latest
env:
ARM_CLIENT_ID: ${{ secrets.CLIENT_ID }}
ARM_CLIENT_SECRET: ${{ secrets.GH_TERRAFORM_PRD }}
ARM_SUBSCRIPTION_ID: ${{ secrets.SUB_ID }}
ARM_TENANT_ID: ${{ secrets.TENANT_ID }}
ARM_ACCESS_KEY: ${{secrets.GH_TERRAFORM_PRD_STATE_BLOB_KEY}}
steps:
- name: Checkout
uses: actions/checkout@master
- name: "Security Scan"
uses: triat/terraform-security-scan@master
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Lint Code Base
uses: github/super-linter@master
env:
GITHUB_TOKEN: ${{ secrets.GH_TOKEN }}
VALIDATE_ALL_CODEBASE: true
VALIDATE_MD: true
VALIDATE_TERRAFORM: true
- uses: hashicorp/setup-terraform@master
with:
terraform_version: latest
- name: Terraform Format
id: fmt
run: terraform fmt -check
- name: Terraform Init
id: init
run: terraform init
- name: Terraform Validate
id: validate
run: terraform validate -no-color
env:
TF_VAR_github_token: ${{ secrets.GH_TOKEN }}
- name: Terraform Plan
id: plan
run: terraform plan -no-color
continue-on-error: true
env:
TF_VAR_github_token: ${{ secrets.GH_TOKEN }}
- uses: actions/github-script@master
env:
PLAN: "terraform\n${{ steps.plan.outputs.stdout }}"
with:
github-token: ${{ secrets.GH_TOKEN }}
script: |
const output = `#### Terraform Security Scan ☠ \`Success! The configuration is secure.\`
#### Terraform Format and Style 🖌 \`${{ steps.fmt.outcome }}\`
#### Terraform Initialization ⚙️ \`${{ steps.init.outcome }}\`
#### Terraform Validation 🤖 ${{ steps.validate.outputs.stdout }}
#### Terraform Plan 📖 \`${{ steps.plan.outcome }}\`
<details><summary>Show Plan</summary>
\`\`\`${process.env.PLAN}\`\`\`
</details>
*Pusher: @${{ github.actor }}, Action: \`${{ github.event_name }}\`, Working Directory: \`${{ env.tf_actions_working_dir }}\`, Workflow: \`${{ github.workflow }}\`*`;
github.issues.createComment({
issue_number: context.issue.number,
owner: context.repo.owner,
repo: context.repo.repo,
body: output
})
- name: Terraform Plan Status
if: steps.plan.outcome == 'failure'
run: exit 1
On a merge, we run the following workflow:
name: Terraform Apply
on:
push:
branches:
- main
jobs:
terraform:
name: Apply
runs-on: ubuntu-latest
env:
ARM_CLIENT_ID: ${{ secrets.CLIENT_ID }}
ARM_CLIENT_SECRET: ${{ secrets.GH_TERRAFORM_PRD }}
ARM_SUBSCRIPTION_ID: ${{ secrets.SUB_ID }}
ARM_TENANT_ID: ${{ secrets.TENANT_ID }}
ARM_ACCESS_KEY: ${{ secrets.GH_TERRAFORM_PRD_STATE_BLOB_KEY }}
steps:
- name: Checkout
uses: actions/checkout@master
- uses: hashicorp/setup-terraform@master
with:
terraform_version: 0.14.3
- name: Terraform Init
id: init
run: terraform init
- name: Terraform Apply
id: apply
run: terraform apply -auto-approve -input=false
env:
TF_VAR_github_token: ${{ secrets.GH_TOKEN }}
Release Drafter is set up as follows:
name: Release Drafter
on:
push:
branches:
- main
jobs:
update_release_draft:
runs-on: ubuntu-latest
steps:
- uses: release-drafter/release-drafter@master
env:
GITHUB_TOKEN: ${{ secrets.GH_TOKEN }}
with the following YAML for config:
categories:
- title: '🚀 Features'
labels:
- 'feature'
- title: '🐛 Bug Fixes'
labels:
- 'bug'
- title: '🧰 Maintenance'
label: 'maintenance'
- title: '📖 Docs'
label: 'docs'
- title: '⚙️ Dependencies'
label: 'dependencies'
change-template: '- $TITLE @$AUTHOR (#$NUMBER)'
change-title-escapes: '\<*_&' # You can add # and @ to disable mentions, and add ` to disable code blocks.
template: |
## Changes
$CHANGES
To ensure everything is captured in the release notes, we also use the Enforce Label Action:
name: Enforce PR labels
on:
pull_request:
types: [labeled, unlabeled, opened, edited, synchronize]
jobs:
enforce-label:
runs-on: ubuntu-latest
steps:
- uses: yogevbd/enforce-label-action@master
with:
REQUIRED_LABELS_ANY: "bug,docs,feature,maintenance,dependencies"
REQUIRED_LABELS_ANY_DESCRIPTION: "Select at least one label ['bug','docs','feature','maintenance','dependencies']"
This is what you see in the Pull Request Tab, each time the PR workflow runs. Allowing you to quickly see if a PR can be merged:
We also push this information into Microsoft Teams using the GitHub Integration App
DevSecOps
An additional benefit of using a CI workflow is adding automated tests. In this scenario, I’ve added a step leveraging tfsec to scan for static code vulnerabilities. In the example below, tfsec warns against creating an Azure network security rule which is fully open. This will halt and fail the workflow unless I provide an ignore comment to accept the warning.
Summary
In this post, I explored using GitHub Actions as a CI workflow that could build and maintain a GitHub organization including users, teams, permissions, security etc. I started by generating a new GitHub repository, then wrote the GitHub Workflow files, and finally started testing the CI workflow and introducing small, incremental changes.
Using Terraform to manage GitHub organisations can really help your developer teams to onboard quickly and securely.
I hope I could help you learn something new today, and share how we do things here at FirstPort.
Any questions, get in touch on Twitter
Top comments (0)