Automating Django Deployment workflow with Github Actions

If you’ve worked on a project that has constant changes, tests, and needs to be deployed to a server every time the changes occur, then you’re aware how the process of logging in to the server via ssh, redeploying, and testing is quite repetitive and frankly annoying sometimes. Luckily, with the advent of Github actions, this can be automated, leading to a happy developer.

The purpose of this article is to get you introduced to the concept of Github actions for a typical Django app deployment, but it can easily apply to any other framework. So...let’s see some Github actions in action.

Introduction to Github actions

According to the official docs, the definition of GitHub actions is;

GitHub Actions is a continuous integration and continuous delivery (CI/CD) platform that allows you to automate your build, test, and deployment pipeline. You can create workflows that build and test every pull request to your repository or deploy merged pull requests to production.

GitHub Actions goes beyond just DevOps and lets you run workflows when other events happen in your repository.

GitHub provides Linux, Windows, and macOS virtual machines to run your workflows, or you can host your own self-hosted runners in your own data center or cloud infrastructure.

What this basically means is that they provide a Continuous Integration and Continuous Delivery pipeline, all in one platform. This is a powerful tool for dev teams and Individual Contributors and is usually a key step in the software development life cycle.

Key Terms and Concepts In Github Actions

Before we begin, let’s familiarize ourselves with a few terms and concepts that you’ll be seeing along with the post.

Workflow

A workflow is a configurable automated process that will run one or more jobs. Workflows are defined by a YAML file in your repository and will run when triggered by an event in your repository, or they can be triggered manually, or at a defined schedule.

Your repository can have multiple workflows each of which can perform a different set of steps. For example, you can have one workflow to build and test pull requests, another workflow to deploy your application every time a release is created, and still another workflow that adds a label every time someone opens a new issue.

Workflows are contained in the .github/workflows directory.

Events

An event is a specific activity in a repository that triggers a workflow run. For example, an event can be a pull request, push, comment, and so on. For more explanation on this, see events that trigger workflows.

Jobs

A job is a set of steps that execute on your workflow in the same runner. An example of a job would be running unit tests. A step can either be a shell script that will be executed or an action to be run. Steps are executed in order and are dependent on each other. Since each step is executed on the same runner, you can share data from one step to another. For example, you can have a step that builds your application followed by a step that tests the application that was built. which is what we’re going to be doing.

Actions

An action is a custom application for the GitHub Actions platform that performs a complex but frequently repeated task. Actions help reduce the amount of repetitive code that you write in your workflow files. An action can pull your git repository from GitHub, set up the correct toolchain for your build environment, or set up the authentication to your cloud provider.

You can write your own actions, or you can find actions to use in your workflows in the GitHub Marketplace.

Runners

A runner is a server that runs your workflows when they're triggered. Each runner can run a single job at a time. GitHub provides Ubuntu Linux, Microsoft Windows, and macOS runners to run your workflows; each workflow run executes in a fresh, newly-provisioned virtual machine.

Secrets

Secrets are environment variables that your jobs depend on. These are usually sensitive information that you wouldn’t want to be exposed to the public. Such secrets include; secret keys, API keys, access tokens, and passwords.

What you need for this config

Before we set up our first workflow, you’ll need a few things.

1. A Github repository - this is the repo you want to set up your actions. For this demo, you can reference this repo
2. Nektos/act package - it’s always advisable that you test your actions locally before pushing them to Github. This package enables us to do so.
3. A virtual private server - where we’ll be deploying our application. I recommend using digital ocean. They provide the best servers at affordable tiers. Sign up using my affiliate link to get $100 credits for 60 days.
   1. Docker and docker-compose- the nektos/act package runs on docker since we’ll be virtualizing our runners. Install them both on your machine and the server.

Setting Our first workflow

For this tutorial, we need to first understand what our objective is before we write any workflow files.

We want our workflow to;

1. Run our tests every time a pull request is made to the main branch.
2. Deploy to a server every time a push is made to the main branch.

For this setup, we’ll need two workflow files, i.e, tests.yml, and deploy.yml files.

Let’s get started.

Running Tests

On your repo, switch to a new branch, in my case, I have a workflows branch.

Run

mkdir .github && mkdir ./github/workflow && touch ./github/workflows/tests.yml

Open the tests.yml file and paste the following code.

name: Django Tests CI

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

jobs:
  test:
    runs-on: ubuntu-18.04
    strategy:
      max-parallel: 4
      matrix:
        python-version: [3.8, 3.9]

    steps:
      - uses: actions/checkout@v2
      # this fixes local act bug of python setup
      - name: local act python setup fix
        run: |
          # Hack to get setup-python to work on act
          # (see https://github.com/nektos/act/issues/251)
          if [ ! -f "/etc/lsb-release" ] ; then
            echo "DISTRIB_RELEASE=18.04" > /etc/lsb-release
          fi
      - name: Set up Python ${{ matrix.python-version }}
        uses: actions/setup-python@v2
        with:
          python-version: ${{ matrix.python-version }}
      - name: Install Dependencies
        run: |
          python -m pip install --upgrade pip
          pip install -r requirements.txt
      - name: Run Tests
        env:
          DEBUG: ${{ secrets.DEBUG }}
          SECRET_KEY: ${{ secrets.SECRET_KEY }}
          DB_ENGINE: ${{ secrets.DB_ENGINE }}
          DB_NAME: ${{ secrets.DB_NAME }}
          BASE_WEATHER_API_URL: ${{ secrets.BASE_WEATHER_API_URL }}
          WEATHER_API_KEY: ${{ secrets.WEATHER_API_KEY }}
        run: |
          python manage.py test core.tests

Starting from the top, we specify our workflow name

name: Django Tests CI

This can be whatever name you want but always use a name that’s easy to remember and descriptive of what the file does.

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

The on keyword is the entrypoint for our events, in this file, we’d like the action to run every time a push and a pull request is made to the main branch. This way, tests will always run if these scenarios are triggered.

jobs:
  test:
    runs-on: ubuntu-18.04
    strategy:
      max-parallel: 4
      matrix:
        python-version: [3.8, 3.9]

This bit has a few configurations.

• We create a job called test.
• we specify that the job will run on ubuntu 18.04 since my server is running on that OS.
• Inside the test job, we specify a strategy matrix. This allows running our tests on a few different python variations. In this case, we want the tests to be run in python 3.8 and python 3.9 environments.
• max-parallel The maximum number of jobs that can run simultaneously when using a matrix job strategy

steps - Next, we define the series of steps the job will follow. Steps are usually sequential, that is, they’re executed one after the other.

 - uses: actions/checkout@v2
      # this fixes local act bug of python setup
      - name: local act python setup fix
        run: |
          # Hack to get setup-python to work on act
          # (see https://github.com/nektos/act/issues/251)
          if [ ! -f "/etc/lsb-release" ] ; then
            echo "DISTRIB_RELEASE=18.04" > /etc/lsb-release
          fi

The first step is a caveat to note when working with act in a local environment (for mac users). It’ll likely throw an error that there’s a problem setting up python. The command is a bash script that resolves this issue. You can ignore it if you won’t face such a headwind.

 - name: Set up Python ${{ matrix.python-version }}
        uses: actions/setup-python@v2
        with:
          python-version: ${{ matrix.python-version }}
      - name: Install Dependencies
        run: |
          python -m pip install --upgrade pip
          pip install -r requirements.txt

This next step sets up the python versions specified in the matrix. We install the dependencies and update pip in the bash script. (defined in the run keyword)

    - name: Run Tests
        env:
          DEBUG: ${{ secrets.DEBUG }}
          SECRET_KEY: ${{ secrets.SECRET_KEY }}
          DB_ENGINE: ${{ secrets.DB_ENGINE }}
          DB_NAME: ${{ secrets.DB_NAME }}
          BASE_WEATHER_API_URL: ${{ secrets.BASE_WEATHER_API_URL }}
          WEATHER_API_KEY: ${{ secrets.WEATHER_API_KEY }}
        run: |
          python manage.py test core.tests

Finally, we run our tests.

You have noticed the environmental variables in this action. To create environmental variables in your github repo go to your repository > settings > left-sidebar > secrets

Add whatever secrets you might require.

Since we’ll need to test our actions locally first, create a .secrets file in your project root. I decided to name mine ‘act.secrets’.

For a more detailed explanation of how secrets work, check out the official documentation.

Finally, let’s run the tests workflow by using;

act —secret-file act.secrets

Drumrolls 🥁🥁.... if you set up everything correctly, the output should be like this;

Congratulations, you’ve made your first GitHub action! Next, push the code to GitHub, make a PR to the main branch and check if it’s been triggered. Finally, merge the PR to the main branch. This should also trigger the workflow.

Setting Up AutoDeploy Pipeline

Now that we’ve set up our tests action, we’ll need to pass deploy to our server when we merge into the main branch.

For this, make a deploy.yml file under the workflows folder and paste the following snippet.

name: Django Deploy CD
on:
  push:
    branches: [main]
jobs:
  deploy:
    runs-on: ubuntu-18.04
    steps:
      - name: Deploy to server
      # don't run locally
        if: ${{ !env.ACT }}
        uses: appleboy/ssh-action@master
        with:
          host: ${{ secrets.LIVE_SERVER_IP }}
          username: ${{ secrets.SERVER_USERNAME }}
          password: ${{ secrets.SERVER_USER_PASSWORD }}
          port: ${{ secrets.DEPLOY_PORT }}
          script: |
            cd ${{ secrets.PROJECT_PATH }}
            git pull ${{secrets.REPO_URL}}
            docker-compose down 
            docker-compose up --build -d

Like the first file, this folder follows the same structure, just different commands. I will cover the new parts.

if: ${{ !env.ACT }} - this command instructs the nektos/act package to ignore this job. This is because, in most cases, we don’t want to deploy to our server from our local environment.

uses: appleboy/ssh-action@master - this instructs our job to use the publicly available GitHub action, “appleboy/ssh-action@master” which is an ssh client. Check out their official docs.

   with:
          host: ${{ secrets.LIVE_SERVER_IP }}
          username: ${{ secrets.SERVER_USERNAME }}
          password: ${{ secrets.SERVER_USER_PASSWORD }}
          port: ${{ secrets.DEPLOY_PORT }}

This action requires us to define a few secrets. The final expected GitHub secrets for this job are;

LIVE_SERVER_IP - server ip address
SERVER_USERNAME - user logging into the server
DEPLOY_PORT - ssh port, default is 22
SERVER_USER_PASSWORD - password of user, can using public key as well in place of this
USERNAME= github username
PROJECT_PATH - path the project is located on server
REPO_URL=https://{USERS_GITHUB_TOKEN}@github.com/{USERNAME}/{repo-name}.git

Finally, we run a series of commands once the action has managed to ssh into the server. Since I’m using docker and docker-compose, my pipeline is as follows.

   script: |
            cd ${{ secrets.PROJECT_PATH }}
            git pull ${{secrets.REPO_URL}}
            docker-compose down 
            docker-compose up --build -d

For the final piece, push the code to Github, make a PR to the main branch(remember this will trigger the test workflow). Once the tests have run, merge the PR, and if you followed along fine, the tests and deployment actions will be set in motion.

If you configured everything well, the output should be similar to this.

Test workflow

Deploy Action

Resources

The content of this post can be nerve-wracking and seem a lot to bite, however, if you feel you need further reading on some of the concepts explained, do check out these curated resources I used while doing my research.

1. The official Github actions documentation is a treasure trove on all things Github actions.
2. Colby Fayock’s article on Freecodecamp.
3. This tutorial on Brad Traversy’s Youtube channel.

Conclusion

Github actions offer a way for DevOps teams to quickly set up a CI/CD pipeline. In this post, we set up a Django application, with some automated tests. We also set up two GitHub actions; one that automatically runs our tests when a pull request is made to the main branch, and another that automatically deploys the application to a virtual private server.

You can read further on working with different environments, setting up restrictions on certain actions and deploying your own actions to the Github marketplace.

Thanks for following along this far and I hope this goes a long way to aid in saving your precious time by avoiding manually doing this.

If you have any thoughts, extra pointers, or just a comment, feel free to reach out by leaving a comment, or shooting up a dm on Twitter

Sponsors

• Scraper API is a startup specializing in strategies that'll ease the worry of your IP address from being blocked while web scraping. They utilize IP rotation so you can avoid detection. Boasting over 20 million IP addresses and unlimited bandwidth. Using Scraper API and a tool like 2captcha will give you an edge over other developers. The two can be used together to automate processes. Sign up on Scraper API and use this link to get a 10% discount on your first purchase.
• Do you need a place to host your website or app, Digital ocean is just the solution you need, sign up on digital ocean using this link and experience the best cloud service, provider.
• The journey to becoming a developer can be long and tormentous, luckily Pluralsight makes it easier to learn. They offer a wide range of courses, with top quality trainers, whom I can personally vouch for. Sign up using this link and get a 50% discount on your first course.