DEV Community

TekWizely
TekWizely

Posted on

Running Your BATS Tests Against Multiple Bash Versions in GitHub

In this post I'll go over the github workflow I created that allows me to test bash scripts using BATS against multiple versions of Bash.

TL;DR: See my github workflow : bats-multi-bash.yml

My Bash Script

The script in question is my Bash-TPL project, which is a light-weight shell-scripting template engine.

My Testing Framework

I chose BATS as the testing framework, as its comprehensive, tests are easy to write, and its Bash all the way down !

I had already written an extensive suite of tests, so I was confident they would reveal any compatibility issues once I figured out how to run them against older Bash versions.

Finding Older Bash Versions

In researching how I was going to run my tests against older bash versions, I discovered the Official Bash Docker Repository

Local Testing

This made running the tests on a specific bash version very easy to do on my local machine:

$ docker --rm -it -v"${PWD}:/bash-tpl" bash:3.2

bash-3.2# apk add bats diffutils
# ...
OK: 8 MiB in 21 packages

bash-3.2# cd /bash-tpl
bash-3.2# bats test

1..92
ok 1 BASH_TPL_TAG_DELIMS: Should do nothing when unset or empty
ok 2 BASH_TPL_TAG_DELIMS: Should error on invalid input
ok 3 BASH_TPL_TAG_DELIMS: Should process valid input
# OK looking good !
# ...
# DOH!
not ok 42 misc-function: escape_regex
not ok 43 misc-function: normalize_directive
# ...
Enter fullscreen mode Exit fullscreen mode

Hate Compatibility Much?

So it turned out that my script only worked on Bash 5.1 :(

Checking Multiple Bash Versions

Once I identified all of the compatibility issues, I was able to work on fixing them, using the above technique to test against multiple Bash versions. Specifically, I ran tests against:

  • Bash Version 3.2 (bash:3.2)
  • Bash Version 4.4 (bash:4.4)
  • Bash Version 5.0 (bash:5.0)
  • Bash Version 5.1 (bash:5.1)

Auto-Detecting Future Issues

Having successfully fixed the compatibility issues, and making a new release, I set out to create a github workflow to run these tests for me in the future.

Workflow Research

Bash Docker Images

I had to determine if I could utilize the previously discovered Bash docker images within a github workflow.

Run a Single Bash Image

My first goal was to determine if I could run the workflow within just a single docker image.

Thanks to the well-documented Workflow Syntax For Github Guide, I discovered the jobs.<job_id>.container configuration.

jobs:
  my_job:
    container:
      image: "bash:3.2"
Enter fullscreen mode Exit fullscreen mode

This will run your job steps inside the specified container !

Configuring BATS

Next I needed to install BATS into my fresh new Bash container.

I knew I could issue an apk add bats command, but I wanted to have more control over the actual version of BATS being used.

GitHub Actions Marketplace

I figured I probably wasn't the first person who wanted to run BATS in a workflow, so I searched the github actions marketplace.

Sure enough, there was already an action for configuring BATS:

Most importantly, it allows you to declare which version of BATS you want to install.

For me, that was the shiny new 1.5.0:

jobs:
  my_job:
    steps:
      - name: Setup BATS
        uses: mig4/setup-bats@v1
        with:
          bats-version: 1.5.0
Enter fullscreen mode Exit fullscreen mode

Multiple Bash Versions

Now that I could successfully run my tests using a single container, it was time to figure out how to use multiple containers.

Enter The Matrix

From the github workflow documentation of the jobs.<job_id>.strategy.matrix strategy:

You can define a matrix of different job configurations. A matrix allows you to create multiple jobs by performing variable substitution in a single job definition. For example, you can use a matrix to create jobs for more than one supported version of a programming language, operating system, or tool. A matrix reuses the job's configuration and creates a job for each matrix you configure.

Not Exactly New

I first learned about the github workflow matrix strategy when implementing github-repo-stats to save stat history for all of my repositories.

But What About Containers?

But I couldn't find any examples of a matrix being used for container setup.

... Favors The Bold

I decided to give a try anyway - And It Worked !

jobs:
  my_job:
    strategy:
      matrix:
        bash: ["bash:3.2", "bash:4.4", "bash:5.0", "bash:5.1"]
    container: 
      image: ${{ matrix.bash }}
Enter fullscreen mode Exit fullscreen mode

Putting It All Together

Now that I had all the pieces worked out, I could finally implement my workflow.

Below is the version of the workflow at the time of posting:

bats-multi-bash.yml

name: BATS Tests

on:
  push:
    branches: [ main ]
  pull_request:
    branches: [ main ]

  workflow_dispatch:

jobs:
  bats:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        bash: ["bash:3.2", "bash:4.4", "bash:5.0", "bash:5.1"]
    container: 
      image: ${{ matrix.bash }}
    steps:
      - name: Install packages
        run: apk add diffutils

      - name: Setup BATS
        uses: mig4/setup-bats@v1.2.0
        with:
          bats-version: 1.5.0

      - name: Checkout code
        uses: actions/checkout@v2

      - name: Run bash-tpl tests
        run: bats test/

      - name: Run template tests
        run: bats test/tpl
Enter fullscreen mode Exit fullscreen mode

Results

The BATS tests run successfully on all specified Bash versions, and they're fast!

BATS Test Results Against Multiple Bash Versions

Conclusion

Thank you for reading - I hope you found this post helpful. Please let me know if you have any comments or questions.

-TekWizely

Discussion (0)