DEV Community

Cover image for Creating a CI/CD Pipeline with Docker and GitHub Actions: A Step-by-Step Guide
Akos
Akos

Posted on • Originally published at akoskm.com on

Creating a CI/CD Pipeline with Docker and GitHub Actions: A Step-by-Step Guide

We all love shipping new features and building stuff probably the main reason we're programmers!

And web development doesn't have to be any more complicated unless... something breaks.

When something breaks, it can bite you for two reasons:

  • You must find what broke and when this is quite challenging. It could be that the thing that doesn't work now has been broken for weeks or even months, but you just realized it!

  • Bugs cost you money imagine launching some Cyber Monday deals for the users of your fresh SaaS with the payment system broken.

As someone who has experienced both issues, setting up a CI/CD pipeline is one of the first tasks I undertake for projects I intend to launch.

But what is CI/CD, and why are they almost always mentioned together?

Terminology

CI

CI stands for Continuous integrations. When you change your application code, you'd like it to work with the rest of the code. CI helps you verify this.

Without automation, you'd test all app functionality manually before shipping the change to the users. With automation, these changes are tested as part of CI.

Running a linter or the TypeScript compiler is a great way to start your CI workflow. They catch early bugs fast and prevent your more resource-intensive steps, such as unit or integration tests, from running.

Once the CI step is done, you're confident your new code works with the rest of the code.

CD

CD is showtime. 🚀 It's time to ship this change to your users, usually by deploying the app on a remote server. But for desktop applications, it means packaging and uploading the executables to storage.

Because of this, I use CD as an acronym, Continuous Delivery, and not Deployment not everything is a web app and can be deployed.

Whatever you do, at this point, CI has already given you the confidence that you're ready to move forward and introduce this change to your users.

GitHub Actions

GitHub Actions is a feature of GitHub for CI/CD within your GitHub repository. It allows you to create custom workflows to build, test, package, release, or deploy your code based on specific triggers such as a push event, a pull request, commits, etc. Workflows consist of one or more jobs , which are series of steps that run commands.

Adding CI to your existing project

First, we'll build a simple CI workflow that will run tsc when we make a commit. Creating a workflow starts with creating a workflow file.

Creating a Workflow file

Jump into your project's root directory and create the file .github/workflows/main.yml:

mkdir -p .github/workflowstouch .github/workflows/main.yml
Enter fullscreen mode Exit fullscreen mode

The first command creates the folder structure (make sure you use -p otherwise you'd have to create the .workflows folder separately) and the second creates an empty file.

Open this file and then proceed to the next step.

Adding steps to your workflow file

There are three elements of a workflow file you need to define:

name: Deploy to Digital Oceanon:jobs:
Enter fullscreen mode Exit fullscreen mode
  • name the name of your workflow.

  • on a rule that describes when this workflow is triggered.

  • jobs - you can have as many jobs as you want. GitHub Actions jobs run in parallel. This means that if you have multiple jobs defined in your workflow, they will all start running simultaneously when the workflow is triggered. You can also make them dependent on each other with the needs keyword, but we'll explore that next time.

The steps listed in jobs can be many things:

  • single line commands such as copying files or logging parameters

  • running npm, yarn, or any other executable on the system

  • running other GitHub actions

Example of a workflow file

If you want to work with your repository in the workflow and what else you would do anyway the workflow must access it.

We already mentioned that GitHub Actions can run other GitHub actions. Luckily, there's a GitHub action for checking out the current repository's source code. You'll be using it like this:

jobs: deploy: runs-on: ubuntu-latest steps: - name: Checkout code uses: actions/checkout@v4
Enter fullscreen mode Exit fullscreen mode

Now that you have the source code in place, you can do npm install and do some type-checking.

Here's how the workflow will look like that does all this:

name: Deploy to Digital Oceanon: push: branches: - main # Set a branch to deploy when pushedjobs: deploy: runs-on: ubuntu-latest steps: - name: Checkout code uses: actions/checkout@v4 - name: Install run: npm install - name: Run Typecheck run: npm run typecheck
Enter fullscreen mode Exit fullscreen mode

Testing the workflow file

It's time to commit and push these changes to our main branch. Once you open the commit history, you'll notice a appeared next to your commit message.

simple workflow passing in github actions

This indicates that the commit triggered a workflow run. Click the to explore further.

You recognize the steps from our workflow file:

screenshot showing all workflow steps

Great job! You just triggered your workflow run that will automatically check for TypeScript errors every time a commit is made to the main branch!

Server Setup - AWS/DigitalOcean

Now that we have some basic checks in place and the confidence in our codebase has grown, it's time to deploy our application.

Deployment options

There are countless options for where to deploy your application, and new ones are added almost every month.

Choosing a service provider is not an easy task, but I have two go-to solutions:

  • DigitalOcean - I use it most of the time. Suitable for small MVPs as well as for more complex apps. Super versatile and easy to use. Simply pricing, fixed-price instances.

  • AWS - endless solutions, a lot more complex, but it's more feature-rich. DigitalOcean runs on AWS. Pay-as-you-go model - takes some time to understand, but it's flexible.

Luckily, both let you rent a single server instance that you can connect to with SSH and deploy your stuff, set up a database, and other applications that will support your main app, and so on.

Deployment Overview

On a high level, for both providers, you have to do three things:

  1. Generate a pub/private key pair locally

  2. Create a server on the platform EC2 for AWS, Droplet on DigitalOcean

  3. Tell the server to trust the pub key I'll describe this later

  4. SSH into the server and deploy your app using Docker

Create a Server

You'll need a passwordless keypair for both providers.

To create one use ssh-keygen -t ed25519 -a 100 -f ~/.ssh/deploy_key_aws. To make it passwordless, simply don't specify a password when prompted.

AWS

  1. Sign up for AWS

  2. Go to Key pairs and do Import key pair. This is the only way I found to import locally created keypairs to AWS and avoid using their pem keys - that you have to download, store, etc. Click Browse under Key pair file and upload ~/.ssh/deploy_key_aws.pub. Click Import key pair. You can also follow the official docs here.

  3. Create an EC2 Instance, and at the Key pair (login) section, specify the keypair you imported. If you have an existing instance, see the docs here.

  4. Launch the instance

DigitalOcean

  1. Sign up for DigitalOcean with my referral link or by going to www.digitalocean.com. In the sidebar, click Droplets, choose Create Droplet, and follow the wizard. You'll find the detailed instructions on creating your first Droplet from the Control Panel here.

  2. Copy the public key value. You can print it to the console using cat ~/.ssh/deploy_key_aws.pub.

  3. Create a Droplet

  4. Connect to the Droplet using their Console

  5. Open the file .ssh/authorized_keys

  6. Add the value of Step 2. as a new line in this file and save the file

Docker Setup

Neither the DigitalOcean Droplet nor the AWS EC2 instance will have docker installed by default, so you have to follow the official guide to install and set them up correctly:

AWS

This could be a guide for itself, but GitHub Copilot told me these are the steps and they worked.

Log into your AWS EC2 machine and run the following:

sudo yum update -ysudo yum install dockersudo service docker startsudo usermod -a -G docker ec2-usersudo chkconfig docker on
Enter fullscreen mode Exit fullscreen mode

Log out and log back in again to pick up the new docker group permissions. But you don't have to do this because we'll deploy from GitHub Actions anyway.

DigitalOcean

They actually have a guide! 👏

https://www.digitalocean.com/community/tutorials/how-to-install-and-use-docker-on-ubuntu-22-04#step-1-installing-docker

Don't read the whole thing, just Step 1 and Step 2.

Adding CD

Now, you'll connect your GitHub workflow to one of these servers and deploy your application.

You'll use Docker Hub to store the images and Docker to deploy them.

You'll find a detailed guide on Dockerizing your Node.js applications in How to Publish Docker Images to Docker Hub with GitHub Actions .

For this guide, I'll assume you already read the previous guide and successfully dockerized the app you want to deploy.

The previous guide ended with building and pushing the image to Docker Hub, pulling and running it locally.

This time, instead of running it locally, you'll SSH into the AWS EC2 or Digital Ocean Droplet, pull and deploy the app there.

To do this, extend your workflow file .github/workflows/main.yml with the following:

name: Deploy to Digital Oceanon: push: branches: - main # Set a branch to deploy when pushedjobs: deploy: runs-on: ubuntu-latest steps: # this is unchanged # checkout & typescript checks # pushing the image to docker hub - see the link above - name: SSH and deploy to AWS EC2 uses: appleboy/ssh-action@master with: host: ${{ secrets.AWS_EC2_IP }} username: ${{ secrets.AWS_EC2_USER }} key: ${{ secrets.AWS_EC2_SSH_KEY }} script: | docker pull akoskm/saas:latest docker stop saas_container docker rm saas_container docker run -d --name saas_container --env-file .env -p 3000:3000 akoskm/saas:latest
Enter fullscreen mode Exit fullscreen mode

This GitHub action uses appleboy/ssh-action to log into the target machine specified by AWS_EC2_IP and using the credentials AWS_EC2_USER and AWS_EC2_SSH_KEY.

These will be your Action Secrets, which you must create by going to repository Settings. Select Secrets and Variables > Actions in the sidebar and click New repository secret.

adding a repository secret in GitHub

Here's how to find the values for all three secrets:

Testing CD

Push a new commit to the main branch and watch the workflow trigger. Compared to what you were seeing in Testing the workflow file previously, you should have these additional steps:

Complete workflow passing

Congratulations, you just created a fully integrated CI/CD workflow for your application!

proud Ron Gif

Conclusion and further improvements

CI/CD doesn't stop here! While this was enough to get you started, here are some more ideas:

  • include in the workflow your unit and integration tests

  • create a workflow that deploys to a staging server from a development branch and a workflow that deploys to production from main

  • try creating multiple jobs inside the same workflow file. You could create a static-checks job that runs npm install, build, typechecks, and lint, and a deploy job that builds the image and deploys it, but only if static-checks is successful. See jobs.<job_id>.needs.

Good luck with taking your app development workflow to the next level.

I understand this process is not trivial, so if you have any questions, please reach out to me on X @akoskm or in the comments.

Thanks,

Akos

Top comments (0)