Once again, I gave myself a challenge to learn how to use Github Action. I tried to see if it was possible to dynamically launch a job for each folder in a directory.
We often want to run tests in parallel to reduce the time of continuous integration. Often tools like Jest or CucumberJS have parallelization features, but these often face the limits of the power available on the machines on which you want to run them.
tests/
user/
foo.feature
bar.feature
account/
example.feature
live/
a.feature
video.feature
I wondered if it wouldn’t be more efficient to run the tests in parallel directly with the CI tool so that only small runners would be needed. In the previous example, I would like to be able to run in parallel on a separate runner for each folder in test/.
A Solution with Github action
name: CI
on:
push:
branches: [main]
jobs:
directories: # Job that list subdirectories
runs-on: ubuntu-latest
outputs:
dir: ${{ steps.set-dirs.outputs.dir }} # generate output name dir by using inner step output
steps:
- uses: actions/checkout@v2
- id: set-dirs # Give it an id to handle to get step outputs in the outputs key above
run: echo "::set-output name=dir::$(ls -d */ | jq -R -s -c 'split("\n")[:-1]')"
# Define step output named dir base on ls command transformed to JSON thanks to jq
loop:
runs-on: ubuntu-latest
needs: [directories] # Depends on previous job
strategy:
matrix:
dir: ${{fromJson(needs.directories.outputs.dir)}} # List matrix strategy from directories dynamically
steps:
- run: echo ${{matrix.dir}}
It’s up to you to adapt it to your needs!
Top comments (0)