Last week, I posted a tutorial on How to Test a GitHub Action with GitHub Actions. The audience for that post, as well as this one, are maintainers of GitHub Actions. In that post, I explained my automated approach to testing GitHub Actions, including unit testing and integration testing, all within GitHub Actions itself. I assumed in that post that the Action you are testing is implemented in Python, although most of the post is applicable more generally, or can easily be adapted to other languages.
This week's post is a bonus tip related to testing a GitHub Action within GitHub Actions. In last week's post, I assumed that it was possible to fully automate the validation of the results of the Action we are testing. In particular, I had a step that used Python's unittest module to run a set of tests that verified correctness of any files produced or modified by the action. But what if the action produces something that cannot be verified automatically? For example, what if it produces an image or something else that must be visually inspected for correctness?
Table of Contents: The rest of this post is organized as follows:
- Workflow Artifacts to the Rescue
- Real Example from the user-statistician GitHub Action
- Complete Example Workflow: Last week's full workflow example, but with the new step added at the end
- Where You Can Find Me
Workflow Artifacts to the Rescue
I'm assuming that you have unit tests and already have of a workflow that runs your unit tests, such as something like my example from last week. Now let's consider a case where my previous approach to integration testing isn't feasible, such as if the output of the action itself is an image or something else that must be visually inspected. We can replace the integration testing steps of last week's workflow with the following step (or add this as an additional step if you can partially automate validating your integration test results). In this example step, I'm assuming your action produces an svg file that you want to inspect, but this can work with anything.
- name: Upload files produced by action as a workflow artifact for inspection if necessary
uses: actions/upload-artifact@v3
with:
name: name-for-the-artifact
path: path/to/file/filename.svg
Or perhaps your action produces an entire directory of files, then you can use something like:
- name: Upload files produced by action as a workflow artifact for inspection if necessary
uses: actions/upload-artifact@v3
with:
name: name-for-the-artifact
path: path/to/directory
In both of the above cases, I've used the actions/upload-artifact action to upload either a file or an entire directory of files as a workflow artifact. In both cases, that action will zip up what the path input points to into a zip file with name based on the name input, such as name-for-the-artifact.zip in this example. It is then attached to the run of the workflow that produced it.
In this way, you can download from the Actions tab of your repository, and inspect the results. Just go to the Actions tab, and find the relevant workflow run, such as from a PR or push.
Real Example
I use this approach in the user-statistician, which generates an SVG with a detailed summary of your GitHub activity, such as contributions, language statistics, etc. I have unit tests implemented in Python with the unittest module. But, the SVG as a whole can only really be validated visually, such as to inspect that content fits appropriately, image dimensions are correct based on content, etc.
The last step of my workflow that runs the tests for that project is the following:
- name: Upload generated SVG as a workflow artifact for inspection if necessary
uses: actions/upload-artifact@v3
with:
name: generated-image
path: images/userstats.svg
After the workflow runs, such as while reviewing a PR, I can navigate to the Actions tab of the repository, find the workflow run, and I'll see the following (screenshot of an actual run).
Be aware that if you are using this approach in a private repository that workflow artifacts count against your storage limit. In public repositories, they do not count against your quota, but they are removed after some time period. The default is 90 days.
The complete workflow for this project is found at: build.yml. The repository itself is:
cicirello
/
user-statistician
Generate a GitHub stats SVG for your GitHub Profile README in GitHub Actions
user-statistician
Check out all of our GitHub Actions: https://actions.cicirello.org/
About user-statistician 
| GitHub Actions |
 |
|---|---|
| Build Status |
  
|
| Source Info |
 
|
| Contributors |
 
|
| Support |
  
|
The cicirello/user-statistician GitHub Action generates a detailed visual summary of your activity on GitHub in the form of an SVG suitable to display on your GitHub Profile README Although the intended use-case is to generate an SVG image for your GitHub Profile README you can also potentially link to the image from a personal website, or from anywhere else where you'd like to share a summary of your activity on GitHub. The SVG that the action generates includes statistics for the repositories that you own, your contribution statistics (e.g., commits, issues, PRs, etc), as well as the distribution of languages within public repositories that you own The user stats image can be customized, including the colors such as with one of the built-in themes or your own set of custom…
Complete Example Workflow
Here's last week's full example workflow for automated testing, but with the example artifact upload step added at the end.
name: build
on:
push:
branches: [ main ]
pull_request:
branches: [ main ]
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Setup Python
uses: actions/setup-python@v4
with:
python-version: '3.10'
- name: Run Python unit tests
run: python3 -u -m unittest tests/tests.py
- name: Verify that the Docker image for the action builds
run: docker build . --file Dockerfile
- name: Integration test 1
uses: ./
with:
input-one: something
input-two: true
- name: Integration test 2
uses: ./
with:
input-one: something else
input-two: false
- name: Verify integration test results
run: python3 -u -m unittest tests/integration.py
- name: Upload files produced by action as a workflow artifact for inspection if necessary
uses: actions/upload-artifact@v3
with:
name: name-for-the-artifact
path: path/to/file/filename.svg
Where You Can Find Me
Follow me here on DEV:
Follow me on GitHub:
Vincent A Cicirello
| Sites where you can find me or my work | |
|---|---|
| Web and social media |
  
|
| Software development |
   
|
| Publications |
      
|
If you want to generate the equivalent to the above for your own GitHub profile, check out the cicirello/user-statistician GitHub Action.
Or visit my website:
Vincent A. Cicirello - Professor of Computer Science
Vincent A. Cicirello - Professor of Computer Science at Stockton University - is a researcher in artificial intelligence, evolutionary computation, swarm intelligence, and computational intelligence, with a Ph.D. in Robotics from Carnegie Mellon University. He is an ACM Senior Member, IEEE Senior Member, AAAI Life Member, EAI Distinguished Member, and SIAM Member.
cicirello.org
Top comments (0)