My HNG Journey. Stage Four: Mastering Multi-Environment Deployments: A Deep Dive into CI/CD with Next.js, Docker, and Nginx

#nextjs #github #nginx #githubactions

In Stage Four of my HNG journey, we were grouped into teams, some teams deployed and maintained backend APIs, our team, named Frontend Devops, consisting of brilliant Engineers like AugustHottie , CodeReaper , DrInTech and Susan. We were tasked with the deployment of a Next.js frontend application. The complexity of this stage was heightened by the need to deploy the same application multiple times to connect to each of the different backend APIs handled by other teams. Our solution involved a multi-environment deployment strategy using Docker, Github Environments, GitHub Actions, and Nginx. This article provides a comprehensive look at how we achieved this, including detailed explanations of the scripts and workflows we used.

Introduction

Our objective was to deploy a Next.js application across several environments, each with its own backend API. We achieved this using GitHub Actions for CI/CD, Docker for containerization, and Nginx as a reverse proxy. This stage involved creating a robust deployment pipeline that allowed us to manage multiple environments efficiently.

Prerequisites
To follow along, before diving into the specifics, ensure you have:

A Next.js application.
Docker and Docker Compose installed on your server.
A Linux server configured for hosting Docker containers.
Nginx installed and configured for reverse proxy.
GitHub repository with configured secrets for deployment.

Step 1

Preparing the Deployment Script
We created a deployment script team_deploy.sh located in scripts/team_deploy/. This script was crucial for automating the deployment process for each environment.

team_deploy.sh

#!/bin/bash

set -e

# Check if the team name and port number are provided
if [ -z "$1" ] || [ -z "$2" ]; then
  echo "Error: Team name and port number are required."
  echo "Usage: $0 [team name] [port]"
  exit 1
fi

TEAM_NAME=$1
export PORT=$2

# Navigate to the repository root and pull the latest changes
cd "$(git rev-parse --show-toplevel)"
git pull origin dev

# Pull the latest Docker image
docker pull docker-image:dev

# Deploy the application using Docker Compose
docker compose --project-name $TEAM_NAME -f docker/team-deploy/docker-compose.yml up -d

Explanation:

Check Inputs: Ensures both the team name and port number are provided. If not, the script exits with an error. Navigate to Repository: Uses git rev-parse to find the top-level directory of the repository and updates it with git pull.
Pull Docker Image: Retrieves the latest Docker image from the registry.
Deploy with Docker Compose: Uses Docker Compose to deploy the container, specifying a project name based on the team.

Step 2

Configuring the GitHub Actions Workflow
We set up a GitHub Actions workflow to automate the integration and deployment process. The integration workflow is triggered on every pull request while the deployment workflow was triggered upon the completion of the build and push workflow for docker images gotten from the marketplace. It used the appleboy/ssh-action to execute the deployment script on the server.

.github/workflows/build-lint-test.yml

name: Build, Lint and Test

on:
  pull_request

jobs:
  build_lint_test:
    runs-on: ubuntu-latest

    steps:
      - name: Checkout code
        uses: actions/checkout@v3

      - name: Set up Node.js
        uses: actions/setup-node@v3
        with:
          node-version: "20"

      - name: Cache pnpm modules
        uses: actions/cache@v3
        with:
          path: ~/.pnpm-store
          key: ${{ runner.os }}-pnpm-${{ hashFiles('**/pnpm-lock.yaml') }}
          restore-keys: |
            ${{ runner.os }}-pnpm-

      - name: Install pnpm
        uses: pnpm/action-setup@v4
        with:
          version: 9

      - name: Install dependencies
        run: pnpm install

      - name: Lint code
        run: pnpm lint

      - name: Build email
        run: pnpm email:build

      - name: Build project
        run: pnpm build

      - name: Run tests
        run: pnpm run test:ci

.github/workflows/deploy.yml

name: Team Deployment

on:
  workflow_run:
    workflows: ["Build and Push"]
    types:
      - completed

jobs:
  team-1:
    if: github.event.repository.fork == false
    runs-on: ubuntu-latest

    environment:
      name: "team-1"
      url: ${{ vars.URL }}

    steps:
      - name: Deploy to team-1 environment
        uses: appleboy/ssh-action@v1.0.3
        with:
          host: ${{ secrets.HOST }}
          username: ${{ secrets.USERNAME }}
          password: ${{ secrets.PASSWORD }}
          script: |
            cd repo
            ./scripts/team_deploy.sh team-1 ${{ vars.PORT }}

  ---

  team-7:
    if: github.event.repository.fork == false
    runs-on: ubuntu-latest

    environment:
      name: "team-7"
      url: ${{ vars.URL }}

    steps:
      - name: Deploy to team-7 environment
        uses: appleboy/ssh-action@v1.0.3
        with:
          host: ${{ secrets.HOST }}
          username: ${{ secrets.USERNAME }}
          password: ${{ secrets.PASSWORD }}
          script: |
            cd repo
            ./scripts/team_deploy.sh team-7 ${{ vars.PORT }}

Explanation:

Trigger: The workflow is triggered when the “Build and Push” workflow completes.
Checkout Code: Retrieves the latest code from the repository.
Deploy: Uses the appleboy/ssh-action to SSH into the server and execute the team_deploy.sh script with the environment name and port.

With the implementation of Github Environments we were able to set up multiple environments that corresponds to multiple users in our server. This approach allows each team-deploy job to target individual users on our server enabling the multiple deployment of the next.js application. Big thanks to my mentor Destiny for his brilliance on this.

Step 3

Docker Compose Configuration
We needed just one Docker Compose file to handle the deployment for each environment. This file specified the Docker image and configuration for the application.

docker-compose.yml

version: '3'
services:
  web:
    image: docker-image:dev
    ports:
      - "${PORT}:80"
    volumes:
      - .env:/app/.env

Explanation:
This docker-compose is what is being referenced by the script team_deploy.sh. It uses the exposed port passed into the script as a parameter as the listening port on the host system for the docker container. Now to connect each version of the same frontend to different backend apis, we mount a .env file not being tracked by github into each container. The .env contains the api url and other sensitive configuration strings necessary for each frontend to have unique backends.

Step 4

Setting Up Nginx as a Reverse Proxy
Nginx was configured to route traffic to the appropriate container based on the environment.

/etc/nginx/sites-available/team-1

    server {
        listen 80;
        server_name  team-1.deployment.com;
        location / {
            proxy_pass http://localhost:<team-1-port>;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto $scheme;
        }

Explanation:

This nginx configuration would be replicated across the different teams in your environment with the only difference being the url and port the team's container is listening on the host system.

Step 5

Implementing Docker Cleanup

To keep the server clean, we set up a cron job to periodically remove unused Docker images.

Create a Cron Job

sudo crontab -e
Add the following line to run docker system prune every 2 hours:
bash
Copy code
0 */2 * * * /usr/bin/docker system prune -af

Explanation:

Cron Job: Runs the docker system prune -af command every 2 hours to remove unused containers, networks, and images.

Conclusion
In Stage Four, we successfully deployed the Next.js application across multiple environments using a structured approach involving GitHub Actions, Docker, and Nginx. The deployment script and GitHub Actions workflow automated the process, while Docker Compose and Nginx ensured smooth and efficient service management. The implementation of regular Docker cleanup maintained server performance and reliability.

This stage demonstrated the power of automation in modern deployment practices and highlighted the importance of managing multiple environments effectively. As we advance in our journey, these practices will serve as a foundation for handling even more complex deployment scenarios.

Thank you for following along with this stage of my HNG journey. Each stage has brought its own set of challenges and learning opportunities, and I look forward to continuing this journey and exploring new horizons in deployment and automation 🚀.