CI/ CD with Github Actions

This post was originally published at thbe.org.

My traditional CI/CD pipeline I have used so far was primarily a combination of Github in conjunction with TravisCI. This combination worked pretty well for me but had some downsides. Based on the fact that I use a PRO account on Github I have to deal with public and private repositories. Unfortunately, the pipeline works only with public Github repositories but not with private Github repositories. Also, the way I handled secrets/ passwords that are required in the CD pipeline to deploy code to a third-party platform was not really satisfying me as it was completely manual and required personal hardware. However, when I saw that Github introduced Actions I thought it was worth a try to check it out and to play a little bit around with the new CI/CD pipeline. In the following article, I will share my experience with Github Actions as well as what is good and whatnot.

According to Github, Actions should automate your workflow from idea to production. Sounds good and is something I was looking for. But describing something on an advertising page looks, in reality, most likely somewhat different. So I picked one of the more complicated repositories I use that didn't work before and tried to establish a CI/ CD pipeline based on Github Actions. The candidate I had chosen was my website, more precise, the tools I use to build my website. As I already posted earlier I'm using Jekyll to build my site. All required actions to build and test my site are collected in a Ruby-based Rakefile. For example, when my site is compiled, I run HTML-Proofer to check the HTML code. Additional complexity was added by the fact that my website repository is private due to some parts I need to refactor first before I can release the code.

The first thing that is required is a workflow file. You need to create a directory called .github/workflows in your project root. Inside that directory, you have to create a YAML file. In my case, I named the first file development.yml but you can name it as you like as long as the file ends with ".yml". This file contains the job and the associated actions the build pipeline will execute to test and deploy the code. The following file contains the CI part of the story, this means, it builds the website and check it for errors:

name: Development Workflow

on: [push]

jobs:
  test:
    name: CI Pipeline
    runs-on: ubuntu-latest
    steps:
      - name: Checkout master
        uses: actions/checkout@v1
      - name: Set up Ruby 2.6
        uses: actions/setup-ruby@v1
        with:
          ruby-version: 2.6.x
      - name: Setup bundler and required gems
        run: |
          gem install bundler
          bundle install --jobs 4 --retry 3
      - name: Build and test the website
        run: bundle exec rake

So, what does exactly happens in the configuration file? The first keyword defines the name of the workflow:

The second keyword "on" defines that the workflow should be executed every time I push code into my Github repository. The "jobs" keyword describes what should be executed in the workflow. You can define one or more jobs, depending on what you would like to execute.

The first version of the development.yml file I've shown before starts a container based on ubuntu-latest. After the start of the container, the configuration file defines four actions that should be executed inside the Ubuntu container. The first action that takes place is the checkout of my current git repository. The second action is to set up Ruby with version 2.6. Action number three setup the required Ruby packages based on the Gemfile in my repository root. When these three actions have been performed, the fourth action can do the effective test of my web site based on the Rakefile that is also defined in my git repository root.

Once the actions have been performed you can see the result in the Actions tab:

You can use the triangle to expand the performed steps to see the logs in detail.

Once the CI scenario has been set up for testing the changes on the website, it's time to configure the CD part for the QA deployment. So, what are we trying to achieve? Easily spoken, when the git commit has passed all required tests I would like to upload the generated site into my test tenant. This is a little bit tricky. I already wrote in my first post about Github Actions that my website is not hosted somewhere in the Github space, instead, it is hosted by a third-party provider in my web-space area. The third-party provider offers an SFTP option to upload the site data. This completely fine when the upload part is done manually but not that easy anymore when a CD tool should do the upload. Fortunately, Github Actions has options for this as well. But before we start with this, let's have a look at how my development.yml looks now with the CD part included:

name: Development Workflow

on: [push]

jobs:
  test:
    name: CI Pipeline
    runs-on: ubuntu-latest
    steps:
      - name: Checkout master
        uses: actions/checkout@v1
      - name: Set up Ruby 2.6
        uses: actions/setup-ruby@v1
        with:
          ruby-version: 2.6.x
      - name: Setup bundler and required gems
        run: |
          gem install bundler
          bundle install --jobs 4 --retry 3
      - name: Build and test the website
        run: bundle exec rake

  deploy:
    name: CD Pipeline QA
    needs: test
    runs-on: ubuntu-latest
    steps:
      - name: webfactory/ssh-agent
        uses: webfactory/ssh-agent@v0.1.1
        with:
          ssh-private-key: ${{ secrets.SSH_PRIVATE_KEY }}
      - name: Checkout master
        uses: actions/checkout@v1
      - name: Set up Ruby 2.6
        uses: actions/setup-ruby@v1
        with:
          ruby-version: 2.6.x
      - name: Set up lftp
        run: sudo apt-get install lftp -y
      - name: Setup bundler and required gems
        run: |
          gem install bundler
          bundle install --jobs 4 --retry 3
      - name: Build and deploy the website to QA
        run: bundle exec rake deploy:qa
        env:
          SSH_DEPLOY_SERVER: ${{ secrets.SSH_DEPLOY_SERVER }}
          SSH_DEPLOY_USER: ${{ secrets.SSH_DEPLOY_USER }}

Compared to the initial YAML I've described in my former post, you might have noticed that I have added a second job called deploy. The actions in that job are pretty much the same as in the test job. This job should also create a new Ubuntu container, it should install Ruby and the Gems required to create my Jekyll site. Two things, however, differ from the test job, the first one is an action called "webfactory/ssh-agent", the second one is a deploy action that uses a special environment. Let's start with the second one, as mentioned, I use a special environment for this action. The rationale behind this is, that I need to pass the target and the credentials to deploy the generated site to this target. To achieve this I created three secrets in the Github secret area of the repository:

In the end, these secrets are "only" variables that will be made available in the container that is started with Github Actions. Nonetheless, this is the missing feature to generate deploy scripts without storing secrets in the code that is pushed to Github. Instead, the secrets are kept outside of the container in a secure area at Github ready to be used at each deploy. In detail this means, I use environment variables in my Rakefile to define the target where I would like to deploy the generated site as well as the user that should be used for the deploy. In principle, it would be possible as well to deploy a password with that way but I made the decision to use a cryptographic key instead.

To make things easier for me I used a predefined action called ssh-agent. This action reads the cryptographic key stored in SSH_PRIVATE_KEY and adds it to the ssh-agent installed in the container. With this mechanism in place, I'm able to deploy the generated site code with a passwordless login.

Last but not least, I need to deploy the generated website code to my production environment, at least when I'm happy with the results from the QA deploy. Till now, the trigger to start the deployment was the push of a commit. When we talk about the productive deployment it makes sense to introduce another flag that must be set before something is deployed to production. Fortunately here comes git into the game. Git allows us to tag a specific commit. Tagging a specific commit in a defined way is the flag I use for the production deployment. Here you see the release.yml that I use for my production deployments:

name: Release Workflow

on:
  push:
    tags:
      - "v*" # Push events to matching v*, i.e. v1.0

jobs:
  deploy:
    name: CD Pipeline PRD
    needs: test
    runs-on: ubuntu-latest
    steps:
      - name: webfactory/ssh-agent
        uses: webfactory/ssh-agent@v0.1.1
        with:
          ssh-private-key: ${{ secrets.SSH_PRIVATE_KEY }}
      - name: Checkout master
        uses: actions/checkout@v1
      - name: Set up Ruby 2.6
        uses: actions/setup-ruby@v1
        with:
          ruby-version: 2.6.x
      - name: Set up lftp
        run: sudo apt-get install lftp -y
      - name: Setup bundler and required gems
        run: |
          gem install bundler
          bundle install --jobs 4 --retry 3
      - name: Build and deploy the website to PRD
        run: bundle exec rake deploy:production
        env:
          SSH_DEPLOY_SERVER: ${{ secrets.SSH_DEPLOY_SERVER }}
          SSH_DEPLOY_USER: ${{ secrets.SSH_DEPLOY_USER }}

The most import thing is the "on -> push -> tags" part. I still deploy with the push event, but only if the commit is tagged starting with a "v". So when I tag my commit as "v1.0.2" it will be automatically pushed to production. The even better part of this solution is, I don't need a special development environment to push a commit to production, it could be completely done from the Github web interface. To do so, you need to go to the releases tab of your code:

Create a release, e.g. v1.0.0:

As soon as the version tag is pushed to the Github platform, the whole CI/ CD chain starts. This means, the development.yml and release.yml instructions run in parallel and, when everything went well and no errors were detected, the web-site goes straight into production:

The release automatically creates the corresponding tag:

This triggers finally the whole CI/ CD pipeline:

And finally, everything is in production as planned:

I hope you enjoyed my three posts on how to set up and how to use Github Actions. So far I'm quite surprised how flawless and good the CI/ CD pipeline already works and I didn't find any major caveats so far that prevents me from using Github Actions. I'm already quite eager to migrate some of my other repositories as well to get more inside into what is possible on Github Actions and whatnot, but for now, that's all I could tell. I'll keep you updated when I had migrated more projects.