Recently I noticed that Mathworks has released a set of GitHub actions for running short Matlab scripts and unit tests. Last year I considered adding CI support for one of my Matlab project but trying to work out how to setup the Matlab license proved too difficult for me at the time. Now with the new actions, I thought: Why not try setting up CI again?
This post describes the steps I took to learn the basics of GitHub Actions and add CI to one of my Matlab projects. I've never used GitHub Actions before, so what better time than during the Actions hackathon! The project I choose to add Actions to is a branch of the Optical Tweezers Toolbox.
This step was relatively simple: I stared by going to the Matlab tests example and copying the GitHub hosted tests example into a new action on my repository. In my development branch I created a file
name: Unit Tests # Controls when the workflow will run # Add [skip ci] to the commit to skip running this job on: push: # My development branch was called draft162 branches: [ draft162 ] jobs: test: name: Run MATLAB Tests and Generate Artifacts runs-on: ubuntu-latest steps: - name: Check out repository uses: actions/checkout@v2 - name: Set up MATLAB uses: matlab-actions/setup-matlab@v1 - name: Run tests and generate artifacts uses: matlab-actions/run-tests@v1 with: test-results-junit: test-results/results.xml code-coverage-cobertura: code-coverage/coverage.xml # I already had a folder called `tests` containing my # existing Matlab unit tests that I would run locally. # Each test file matches the format: test*.m select-by-folder: tests
As soon as I pushed the commit the action started running! No additional steps to setup Matlab! No additional third party tools required!
The Action seems to use a Sponsored Matlab license, although I have no idea what this means. The result was that I didn't need to point Matlab at my universities Matlab license, which made things much easier to setup/run.
After about 4 minutes my tests had run and I could view the logs to find out which tests passed and which tests failed. Overall the process was much easier than I expected. However, to view the logs, we need to be able to download the coverage report or display it after the Action runs; and browsing through the action logs isn't very convenient for finding out if the tests were successful. It would be nice if there was some graphical indication of the testing/coverage status (such as a badge or a success/failure count).
I'd seen testing badges displayed on other repositories that used GitHub Actions. After a quick google I found instructions for how to generate the appropriate URL. Now I can easily display the status of the action from within the project readme or elsewhere in the documentation.
Later I discovered that I could also have clicked the "Create Status Badge" button within the test results page to automatically format this URL:
The action included two directives to specify output files for the test-results and coverage-report
test-results-junit: test-results/results.xml code-coverage-cobertura: code-coverage/coverage.xml
In order to view the contents of these files I needed to create an artifact by adding the
upload-artifact action immediately following my testing action, for example:
- name: Upload results artifact uses: actions/upload-artifact@v2 if: always() with: name: test-restuls path: test-results/results.xml
In this action I needed to specify the
if: always() line to force the action to always be run, even if some of the unit tests failed.
I also tried an alternative approach involving adding the
continue-on-error: true statement to the testing action and adding an additional non-zero return status after all the other actions (see this SO for further details). This approach also worked, but it resulted in a slightly more verbose yaml file.
The result of adding the
upload-artifact action was two archives that I could now download from the Action summary page:
I was then able to view these XML files in my favorite viewer.
Downloading the XML files is not the most convenient workflow. It would be useful to have at least a summary in the browser with the current number of passing/failing test. Taking a quick look at the GitHub marketplace, I found a action that displays exactly this (and more)
GitHub Action to Publish Unit Test Results
This GitHub Action analyses Unit Test result files and publishes the results on GitHub. It supports the JUnit XML file format and runs on Linux, macOS and Windows.
- name: Publish Unit Test Results uses: EnricoMi/publish-unit-test-result-action@v1 if: always() with files: test-results/**/*.xml
- name: Publish Unit Test Results uses: EnricoMi/publish-unit-test-result-action/composite@v1 if: always() with: files: test-results/**/*.xml
See the notes on running this action as a composite action if you run it on Windows or macOS.
Also see the notes on supporting pull requests from fork repositories and branches created by Dependabot.
if: always() clause guarantees that this action always runs, even if earlier steps (e.g…
I added this action to my yaml file, just as I did for the upload action previously:
- name: Publish Unit Test Results uses: EnricoMi/publish-unit-test-result-action/composite@v1 if: always() with: files: test-results/results.xml report_individual_runs: "true"
Running this action adds an additional tab to the action summary which has annotations for each failed test.
report_individual_runs option seems to be needed to get GitHub to display an annotation for each test (as opposed to just one annotation for the whole file).
So far I've been able to setup an action to test most of the code, there are still many things that could be improved including:
- Work out how to display coverage reports. I tried using Code Coverage Summary but couldn't work out why it was not running.
- Look into speeding up CI time by running tests only for affected directory
- CI for graphical user interfaces (not yet supported by Activity)
Wacky Wildcards or Maintainer Must-Haves, not sure which fit is better.
Full yaml file (and a couple of extras):
# Action to runs tests on repository name: Unit Tests # Controls when the workflow will run # Add [skip ci] to the commit to skip running on: push: branches: [ draft162 ] jobs: test: name: Run MATLAB Tests and Generate Artifacts runs-on: ubuntu-latest steps: - name: Check out repository uses: actions/checkout@v2 - name: Set up MATLAB uses: matlab-actions/setup-matlab@v1 - name: Run tests and generate artifacts uses: matlab-actions/run-tests@v1 with: test-results-junit: test-results/results.xml code-coverage-cobertura: code-coverage/coverage.xml # Exclude GUI, not yet supported by test runner select-by-folder: > tests/beam; tests/bsc; tests/drag; tests/dynamics; tests/examples; tests/particle; tests/shape; tests/tmatrix; tests/tools; tests/utils # Folders we should run coverage reports on source-folder: +ott; examples - name: Publish Unit Test Results uses: EnricoMi/publish-unit-test-result-action/composite@v1 if: always() with: files: test-results/results.xml report_individual_runs: "true" # Not sure why, but this doesn't seem to work :( - name: Code Coverage Summary Report uses: irongut/CodeCoverageSummary@v1.2.0 if: always() with: filename: code-coverage/coverage.xml format: markdown badge: true - name: Upload results artifact uses: actions/upload-artifact@v2 if: always() with: name: test-restuls path: test-results/results.xml - name: Upload coverage artifact uses: actions/upload-artifact@v2 if: always() with: name: code-coverage path: code-coverage/coverage.xml
This page describes the Actions added to a branch of the Optical Tweezers Toolbox, a toolbox for modelling particles trapped in optical fields. The Actions should hopefully be merged into the master branch sometime later this year.
ott - Optical Tweezers Toolbox
The optical tweezers toolbox can be used to calculate optical forces and torques of particles using the T-matrix formalism in a vector spherical wave basis The toolbox includes codes for calculating T-matrices, beams described by vector spherical wave functions, functions for calculating forces and torques, simple codes for simulating dynamics and examples.
We are currently working on documentation
and we welcome feedback/suggestions/comments. Additional documentation
can be found via the Matlab
help command or in the source code.
Installation and usage
There are several methods for installing the toolbox If using Matlab, the easiest method is to launch Matlab and navigate to Home -> Addons -> Get-Addons and search for "Optical Tweezers Toolbox". Then, simply click the "Add from GitHub" button to automatically download the package and add it to the path Alternatively, you can download the toolbox directly from the GitHub repository or select a…