Automating Testing: My Journey with GitHub Actions, CI Pipelines, and Test Automation

This week, I was tasked with automating testing, linting, and error checking for my ResumeEnhancer project using a GitHub Actions CI workflow. Continuous Integration (CI) automatically tests new code as soon as it's added, helping to catch issues early before they affect the entire project. This ensures the code remains stable, speeds up development, and facilitates smoother collaboration among developers.

CI pipelines are not new to me, as I’ve developed CI/CD pipelines for previous projects. However, last month, while contributing four pull requests to open-source projects, I realized the importance of pipelines in maintaining clean and consistent code across repositories.

In this blog, I’ll share my experience setting up a CI pipeline. I chose GitHub Actions because I’m already using GitHub for source control. GitHub Actions spin up a machine to execute tasks when code is pushed or a pull request is created. To implement CI with GitHub Actions, you define a series of steps to ensure your code meets the required standards.

For my ResumeEnhancer project, I am using:

• Black for code formatting,
• Flake8 for linting,
• Pytest for testing.

Here’s how my CI pipeline looks:

name: ResumeEnhancer CI Pipeline - Test, Lint, and Format

on:
  push:
    branches: ["main"]
  pull_request:
    branches: ["main"]

permissions:
  contents: read

jobs:
  build:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        python-version: ["3.11", "3.12"]

    steps:
      # Check out the repository code
      - name: Checkout code
        uses: actions/checkout@v3

      # Set up Python
      - name: Set up Python ${{ matrix.python-version }}
        uses: actions/setup-python@v2
        with:
          python-version: ${{ matrix.python-version }}

      # Install dependencies
      - name: Install dependencies
        run: |
          python -m pip install --upgrade pip
          pip install -r requirements.txt

      # Make format.sh and lint.sh executable
      - name: Make scripts executable
        run: chmod +x ./format.sh ./lint.sh

      # Run formatting
      - name: Run format.sh
        run: ./format.sh
        continue-on-error: false

      # Run linting
      - name: Run lint.sh
        run: ./lint.sh
        continue-on-error: false

      # Run tests
      - name: Run Tests
        run: |
          pytest

Initially, I included Python versions ["3.10", "3.11", "3.12"], but my CI run failed. After investigating, I discovered that the tomllib module, which I’m using, is built into Python starting from version 3.11. As a result, I had to remove Python 3.10 from the list.

This CI pipeline for the ResumeEnhancer project automatically runs whenever code is pushed to the main branch or a pull request is created. It uses an Ubuntu environment and tests the code on Python versions 3.11 and 3.12. The pipeline performs several steps:

1. Checks out the repository code.
2. Sets up the required Python version.
3. Installs project dependencies.
4. Ensures that the formatting (format.sh) and linting (lint.sh) scripts are executable and error-free.
5. Runs automated tests using pytest.

Here’s the result after successfully setting up the CI pipeline:

While I’ve worked with CI before, setting it up for my project reminded me of how valuable it is. It’s reassuring to know that every time I push code or open a pull request, the pipeline automatically checks formatting, linting, and tests—saving me from manual effort. It’s like having a safety net that ensures the code remains clean and consistent, which is especially helpful when contributing to open-source projects or collaborating with others. CI reduces stress and makes development more efficient.

Testing Out a Classmate’s CI Workflow

To test CI workflows further, I collaborated with my friend Non. I added a test case to his repository, chat-completion-api, to evaluate his CI pipeline, and he did the same for mine. His project also uses Python, but instead of pytest, it uses unittest for testing.

The main difference between unittest and pytest is that unittest is a built-in testing framework with a structured approach, while pytest offers more flexibility and features, making testing simpler and more powerful. Since I’m familiar with pytest, adapting to unittest was straightforward. I wrote two test cases for a function in utils.py, and here’s the PR.

Overall, I enjoyed working with CI. It simplifies development and ensures high-quality, consistent code.