Automatically build Docker images with GitHub Actions

• Introduction
• Setting up a Runner
• Structure of a GitHub Action
  • 1. name
  • 2. on
  • 3. env
  • 4. jobs
• GitHub Action to build Docker Images
  • Process & preconditions
  • 1. Triggers
  • 2. Git Checkout
  • 3. Split the branch name and get the version number
  • 4. Build the image
  • 5. Tag the image
  • 6. Push the images to the registry
  • 7. Remove all build data from the runner
  • 8. All together
• Conclusions

Introduction

When releasing a new version of a web app, this was the process, which might sound familiar to you:

1. I merged everything into the master branch
2. SSH into the deployment server
3. git fetch
4. git checkout
5. git pull
6. Build docker images (this took loooong and once the web app was big enough, it would fail)
7. Then I would have to build the image in my computer or another specific server to avoid the production server crashing
8. Push that to a registry
9. Go back to the server and pull from the registry
10. Realize you did not update an environment variable
11. AGHHHJJJ!!!
12. 

Here's how I do it now:

1. Create a new staging/ or release/ branch from my develop branch. i.e. release/1.0
2. That's it!

Setting up a Runner

Definition: A runner is an instance that runs whatever you set in your GitHub action. From making it print Hello World to building and deploying apps to making you coffee (seriously).

GitHub gives you 2 options:

• Use one of their runners for free up to X minutes every month and then pay for it
• Spin up a self hosted runner and use that for free

If you go with the first option skip this, otherwise continue reading.

Since I already have a home lab and and setting up a runner is quite easy, self hosted the easy route for me to save a few bucks. You can also do this with an old computer or even a raspberry pi you have laying around.

If you want to learn more about self hosted runners look into GitHub's documentation.

This is what you need to set up a self hosted runner:

1. Select the OS you want your runner to be. I'm a fan of Ubuntu and I can pretty much do everything I need with it, so I went with a Linux runner.
2. Make sure you have any necessary packages installed in the runner. For my use case I would need to install Docker first.
3. Go to your GitHub repo, then Settings
4. In the left panel hit Actions and then Runners
5. Select New self-hosted Runner
6. Select the OS you want to use and architecture
7. Follow the terminal commands displayed below your selection to finish setting up your runner. I do not want to post the commands here since GitHub updates them every once in a while, so just follow their instructions over there.
8. Once you finish setting the runner up in your Linux machine you should see it in the Runners page from before and you're ready to use it!

Structure of a GitHub Action

GitHub actions are defined in .yml files.

They live under .github/workflows/ in your repository and will be live once you push them to your master branch.

You can either set them up directly with GitHub's web interface or with your favorite editor and push them to your master branch.

A (simple) GitHub action has 3 main parts:

1. name

Easy, this is just the name you want to give the action to track it later on.

name: My GitHub action name

2. on

This is what you want to trigger the action on. The most common case is to tricker an action on a push to a specific branch or opening a pull request. But there are many triggers.

You can see documentation on all the triggers here:
https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows

Here's an example code to:

• Be able to trigger it manually. For more information, see "Manually running a workflow."
• Be triggered when there's a push to the my-branch-name and my-other-branch-name branches.

on:
  # Trigger the action manually from the UI
  workflow_dispatch:
  # Trigger the action when pushing to certain branches
  push:
    branches:
      - 'my-branch-name'
      - 'my-other-branch-name'

3. env

Here you can set environment variables for a specific Action. This can be use in conjunction with the Variables and Secrets in the GitHub UI.

If there's something you use in multiple steps or jobs it is useful to set it up here so that you only need to change one line.

env:
  DOCKER_IMAGE_NAME: my-image

4. jobs

These are the individual series of steps you want to execute. Each job has a series of sequential steps and every job can be run asynchronously.

Differences between Jobs and Steps

• Jobs are individual, asynchronous tasks. This means that if you have several tasks that can happen asynchronously (at the same time) (i.e.: deploying an app and uploading new documentation) you can put them in separate jobs. If you have several runners you can execute several jobs at the same time.
• Steps are actions that happen synchronously. If you have tasks that need to run one after the other (i.e.: Building a docker image and then pushing it to a Docker registry) set them up in steps instead of jobs.

Sample code:

jobs:
  build_docker_images:
    # Job name that shows in the GitHub UI
    name: Build Docker Images
    # Runner to use
    runs-on: self-hosted

    steps:
    - name: Checkout
      uses: actions/checkout@v3

   ...

You can set up as many jobs and steps as you like in a single action file.

GitHub Action to build Docker Images

In this section I will show you how to build a docker image from a Dockerfile and push it to my private Docker Registry. All by just creating or pushing to a specific branch.

I will create a future article showing how to deploy an app once it's build. Please comment below if you want to be notified.

Process & preconditions

After checking that your self-hosted runner is available and that the structure of a GitHub action is clear, here are my preconditions:

When I push to the release/... branch I want to get the version name (i.e.: release/1.5 → 1.5), then build the Docker image and tag it with that version (i.e.: my-image:1.5).

After this, I want to push the image to a private Docker Registry I self-host and remove all data from the runner.

Here's the process:

1. Trigger the action manually or with a push to a release/... branch
2. Checkout the branch
3. Split the branch name as to only get the version number
4. Build the Docker image
5. Once the image is built, tag it with the version number and the docker registry
6. Push the image to the registry
7. Remove all build data from the runner

Environment variables

In case I ever need to change the registry URL or image name I set the following environment variables:

env:
  DOCKER_IMAGE_NAME: my-image
  DOCKER_REGISTRY_URL: myregistry.domain.com

1. Triggers

I want not only for the action to be triggered on push to release branches but also manually.

name: Build release Docker image

on:
  # Trigger the action manually from the UI
  workflow_dispatch:
  # Trigger the action when I create or push a `release/**` branch
  push:
    branches:
      - 'release/**'

The two * mean that the action will trigger on any branch name that starts with release/. i.e.: release/1.3

2. Git Checkout

Note on Jobs
I will only require one single job since all my steps need to be sequential.

GitHub provides a default library with many steps integrating their service.

We first need to checkout the branch that triggered the action:

jobs:
  build_docker_images:
    # Job name that shows in the GitHub UI
    name: Build Docker Images
    # Runner to use
    runs-on: self-hosted

    steps:
    - name: Checkout
      uses: actions/checkout@v3

3. Split the branch name and get the version number

- name: Get the release version
  # i.e.: release/1.0.0 -> 1.0.0
  id: strip-branch-name
  run: |
  release_version=$(echo "${{ github.ref }}" | sed 's/refs\/heads\/.*\///')
  echo "Building release version $release_version"
  echo "RELEASE_VERSION=$release_version" >> $GITHUB_ENV
  shell: bash

Here I just run some commands in the Ubuntu bash shell:

• release_version=$(echo "${{ github.ref }}" | sed 's/refs\/heads\/.*\///'): Splits the branch name from the first / using sed
• echo "Building release version $release_version": This just prints out in the GitHub actions UI
• echo "RELEASE_VERSION=$release_version" >> $GITHUB_ENV: This sets the RELEASE_VERSION environment variable for this action, to be used in other steps
• shell: bash: Indicates what shell this code needs to run in

4. Build the image

- name: Build the Docker image
  run: docker build . --file Dockerfile --tag $DOCKER_IMAGE_NAME:$RELEASE_VERSION

Here we use the RELEASE_VERSION environment variable defined in the previous step.

5. Tag the image

I want to tag the image two ways here:

• With the version number tag
• With the latest tag

- name: Tag the image for the private registry
  run: docker tag $DOCKER_IMAGE_NAME:$RELEASE_VERSION $DOCKER_REGISTRY_URL/$DOCKER_IMAGE_NAME:$RELEASE_VERSION

- name: Create a latest image as well
  run: docker tag $DOCKER_IMAGE_NAME:$RELEASE_VERSION $DOCKER_REGISTRY_URL/$DOCKER_IMAGE_NAME:latest

6. Push the images to the registry

- name: Push the Docker image with version number
  run: docker push $DOCKER_REGISTRY_URL/$DOCKER_IMAGE_NAME:$RELEASE_VERSION

- name: Push the latest tag
  run: docker push $DOCKER_REGISTRY_URL/$DOCKER_IMAGE_NAME:latest

7. Remove all build data from the runner

This is a step that I do in all of my Actions. This ensures I always have a clean slate.

In this case it consists in removing all the built images.

- name: Remove the Docker image with version number
  run: docker rmi $DOCKER_REGISTRY_URL/$DOCKER_IMAGE_NAME:$RELEASE_VERSION

- name: Remove the Docker image with latest tag
  run: docker rmi $DOCKER_REGISTRY_URL/$DOCKER_IMAGE_NAME:latest

- name: Remove the local image
  run: docker rmi $DOCKER_IMAGE_NAME:$RELEASE_VERSION

Because I release pretty frequently I don't prune Docker, so that next time the build happens fast. If you want to remove absolutely all data you can prune the system by:

- name: Prune Docker
  run: docker system prune -a -f

• -a will prune all unused images, not just dangling ones
• -f will avoid docker prompting for a user confirmation

8. All together

Here's the full action code. You can copy and paste it into your YML file and modify it as needed:

name: Build release Docker image

on:
  # Trigger the action manually from the UI
  workflow_dispatch:
  # Trigger the action when I create or push a `release/**` branch
  push:
    branches:
      - 'release/**'

env:
  DOCKER_IMAGE_NAME: my-image
  DOCKER_REGISTRY_URL: myregistry.domain.com

jobs:
  build_docker_images:
    # Job name that shows in the GitHub UI
    name: Build Docker Images
    # Runner to use
    runs-on: self-hosted

    steps:
    - name: Checkout
      uses: actions/checkout@v3

    - name: Get the release version
      # i.e.: release/1.0.0 -> 1.0.0
      id: strip-branch-name
      run: |
        release_version=$(echo "${{ github.ref }}" | sed 's/refs\/heads\/.*\///')
        echo "Building release version $release_version"
        echo "RELEASE_VERSION=$release_version" >> $GITHUB_ENV
        shell: bash

    # Build the Docker image
    - name: Build the Docker image
      run: docker build . --file Dockerfile --tag $DOCKER_IMAGE_NAME:$RELEASE_VERSION

    # Tag the Docker Images
    - name: Tag the image for the private registry $DOCKER_REGISTRY_URL
      run: docker tag $DOCKER_IMAGE_NAME:$RELEASE_VERSION $DOCKER_REGISTRY_URL/$DOCKER_IMAGE_NAME:$RELEASE_VERSION

    - name: Create a latest image as well
      run: docker tag $DOCKER_IMAGE_NAME:$RELEASE_VERSION $DOCKER_REGISTRY_URL/$DOCKER_IMAGE_NAME:latest

    # Push the images to the registry
    - name: Push the Docker image with version number
      run: docker push $DOCKER_REGISTRY_URL/$DOCKER_IMAGE_NAME:$RELEASE_VERSION

    - name: Push the latest tag
      run: docker push $DOCKER_REGISTRY_URL/$DOCKER_IMAGE_NAME:latest

    # Remove the local images
    - name: Remove the Docker image with version number
      run: docker rmi $DOCKER_REGISTRY_URL/$DOCKER_IMAGE_NAME:$RELEASE_VERSION

    - name: Remove the Docker image with latest tag
      run: docker rmi $DOCKER_REGISTRY_URL/$DOCKER_IMAGE_NAME:latest

    - name: Remove the local image
      run: docker rmi $DOCKER_IMAGE_NAME:$RELEASE_VERSION

Conclusions

With this set up you now automatically will have Docker images ready for use in the Registry of your choice.

I will create a future article showing how to automatically deploy an app once it's build and in a registry. Please comment below if you want to be notified.