Often when we create CI/CD pipelines, we create some steps where the purpose is to setup something for the upcoming steps. For example, you might add a step to install aws cli, install commands with apt-get, restoring build artifact from a cache, and etc.
These setup commands tend to be copy-pasted across different pipelines making them less DRY. However, in many CI/CD tools or platforms such as GitHub Action, we could define an action, much like a function, to extract the repeated steps into a single reusable action. This process where we extract a bunch of steps to make them reusable can be implemented with a composite action in the GitHub platform.
To quickly demonstrate GitHub composite actions, let say we have the following CI workflow under .github/workflows directory that performs a unit test
# Defined in .github/workflows/ci.yml
name: Some CI workflow
...
jobs:
unitTest:
name: Run Unit Test
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
# Setup step to speed-up dependencies installation
# and app compilation
- name: Restore build artifacts
run: ... # run commands to restore app build artifacts
- name: Install app dependencies
run: ... # run commands to install app dependencies
- name: Compile app
run: ... # run commands to compile app
- name: Run test
run: ... # run commands to run unit test
...
You realize later that the step Restore build artifacts can be used for other existing or future workflows but instead of copy-pasting the code to other workflows, you instead create a composite action called cache under .github/actions/cache directory
# Defined in .github/actions/cache/action.yml
name: Cache
description: "Restore build artifacts"
runs:
using: "composite"
steps:
- name: Cache
run: ... # run commands to restore app build artifacts
by this stage we have the following file tree
.
├── .github
│ ├── actions
│ │ └── cache
│ │ └── action.yml
│ │
│ └── workflows
│ └── ci.yml
└── ...
we could reference our defined cache action inside our CI as follows
name: Some CI workflow
...
jobs:
unitTest:
name: Run Unit Test
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
# Reusing the cache action
- name: Restore build artifacts
uses: ./.github/actions/cache
...
So far, composite action works beautifully to make your workflows DRY-er however there is one drawback which GitHub composite action don't natively support (at least by the time of writing this post) which is to run post steps.
Post steps are simply steps that GitHub will run after a workflows has reach its final steps. For the cache example, we might want to run a post step to rebuild the app build artifacts for future CI. We could define this step as another action however this gives us inconvenience as we have to keep remembering to put the rebuild action at certain step in the workflows. How can we do better and let the composite action handle this post steps seamlessly for us?
After doing a bit of research, I came across the following GitHub issue: Support pre and post steps in Composite Actions in the GitHub action runner repo. In the discussion, I found out that we could use a node action to indirectly perform post steps in a composite actions.
To run post steps in composite actions, first we will re-use the with-post-step action that is defined in here and in this post we wouldn't look into the implementation detail of the code.
.github/actions/with-post-step/action.yml
# Copied from: https://github.com/pyTooling/Actions/blob/main/with-post-step/action.yml
name: With post step
description: "Generic JS Action to execute a main command and set a command as a post step."
inputs:
main:
description: "Main command/script."
required: true
post:
description: "Post command/script."
required: true
key:
description: "Name of the state variable used to detect the post step."
required: false
default: POST
runs:
using: "node16"
main: "main.js"
post: "main.js"
.github/actions/with-post-step/main.js
// Ref: https://github.com/pyTooling/Actions/blob/main/with-post-step/main.js
const { spawn } = require("child_process");
const { appendFileSync } = require("fs");
const { EOL } = require("os");
function run(cmd) {
const subprocess = spawn(cmd, { stdio: "inherit", shell: true });
subprocess.on("exit", (exitCode) => {
process.exitCode = exitCode;
});
}
const key = process.env.INPUT_KEY.toUpperCase();
if ( process.env[`STATE_${key}`] !== undefined ) { // Are we in the 'post' step?
run(process.env.INPUT_POST);
} else { // Otherwise, this is the main step
appendFileSync(process.env.GITHUB_STATE, `${key}=true${EOL}`);
run(process.env.INPUT_MAIN);
}
The file tree should now looks something like this
.
├── .github
│ ├── actions
│ │ ├── cache
│ │ | └── action.yml
│ │ └── with-post-step
│ │ ├── action.yml
│ │ └── main.js
│ │
│ └── workflows
│ └── ci.yml
└── ...
once we have with-post-step action defined, go back to cache action.yml definition and use the with-post-step action to define the post steps of the cache
# Defined in .github/actions/cache/action.yml
name: Cache
description: "Restore and rebuild build artifacts"
runs:
using: "composite"
steps:
- name: Cache
uses: ./.github/actions/with-post-steps
with:
main: ... # run commands to restore app build artifacts
post: ... # run commands to rebuild app build artifacts
Once the post steps is in place for the cache action, using the action in any workflow will add an additional post steps when the workflow is running.
If you are interested to try out this yourself, the code in this post can be found in my composite-post-gha repo.
Top comments (0)