Photo by Ruslan Bardash on Unsplash
Scenario
In building and testing a monorepo in CI, there are problems regarding splitting the tasks in order to lower the time of the CI run.
NX Solution
Nx has a feature - distributed task execution to run tasks parallel in CI that helps speed up building and testing a monorepo.
Unlike manually splitting the build and tests into different jobs, in NX distributed execution the tasks are distributed by a automated manager. The user decided the number of workers to give and the manager utilizes them.
TLDR
CI.yaml - Main Section
name: CI
on:
push:
branches:
- main
pull_request:
workflow_dispatch:
env:
NX_CLOUD_DISTRIBUTED_EXECUTION: true
CI.yaml - Job - Main
jobs:
main:
runs-on: ubuntu-latest
if: ${{ github.event_name != 'pull_request' }}
steps:
- uses: actions/checkout@v2
name: Checkout [main]
with:
fetch-depth: 0
- name: Derive appropriate SHAs for base and head for `nx affected` commands
uses: nrwl/nx-set-shas@v2
- uses: pnpm/action-setup@v2.0.1
with:
version: 6.22.2
- uses: actions/setup-node@v2
with:
node-version: '17'
cache: 'pnpm'
- run: pnpm install
- run: pnpm exec nx-cloud start-ci-run
- run: pnpm exec nx affected --target=build --parallel --max-parallel=3
- run: pnpm exec nx affected --target=lint --parallel --max-parallel=2
- run: pnpm exec nx affected --target=test --parallel --max-parallel=2
- name: Clean Agents
if: ${{ always() }}
run: pnpm exec nx-cloud stop-all-agents
CI.yaml - Job - PR
pr:
runs-on: ubuntu-latest
if: ${{ github.event_name == 'pull_request' }}
steps:
- uses: actions/checkout@v2
with:
ref: ${{ github.event.pull_request.head.ref }}
fetch-depth: 0
- name: Derive appropriate SHAs for base and head for `nx affected` commands
uses: nrwl/nx-set-shas@v2
- uses: pnpm/action-setup@v2.0.1
with:
version: 6.22.2
- uses: actions/setup-node@v2
with:
node-version: '17'
cache: 'pnpm'
- run: pnpm install
- run: pnpm exec nx-cloud start-ci-run
- run: pnpm exec nx affected --target=build --parallel --max-parallel=3
- run: pnpm exec nx affected --target=lint --parallel --max-parallel=2
- run: pnpm exec nx affected --target=test --parallel --max-parallel=2
- name: Clean Agents
if: ${{ always() }}
run: pnpm exec nx-cloud stop-all-agents
CI.yaml - Job - Agents
agents:
runs-on: ubuntu-latest
name: Agent - General
timeout-minutes: 15
strategy:
matrix:
agent: [1, 2]
steps:
- uses: actions/checkout@v2'
- uses: pnpm/action-setup@v2.0.1
with:
version: 6.22.2
- uses: actions/setup-node@v2
with:
node-version: '17'
cache: 'pnpm'
- run: pnpm install
- name: Start Nx Agent ${{ matrix.agent }}
run: npx nx-cloud start-agent
Solution Walkthrough
Activating the Distributed Execution Manager
In order for the NX commands to know this they should use the manager for work dispatching, they have to have the environment variable NX_CLOUD_DISTRIBUTED_EXECUTION to true.
We put this variable environment wide for all jobs and steps.
env:
NX_CLOUD_DISTRIBUTED_EXECUTION: true
If we want to disable the usage of distribute tasks for a specific NX command we can set this variable to false temporarily for a specific step for instance.
- name: Publish artifacts
run: pnpm exec nx affected --target=custom-publish --parallel=2 --commitid=$GITHUB_SHA
env:
NX_CLOUD_DISTRIBUTED_EXECUTION: false
NX distributed execution can only run commands that are specified as cachable in the NX comnfig. Commands that are not specified as cachable gives error.
Checking out Branch
To be able to determine which packages were changed - meaning needing build and test - on the main and pr branches, we want to retrieve all the repo commits. Otherwise, NX Affected command will not be able to compare commits regarding packages changes.
We do this by specifying to download all commits in the checkout action.
fetch-depth: 0 - Specify to download all commits
For determine changes of pr branches, we want to checkout not the special branch Github creates which is the merge between the PR branch and the base it is against, but the PR branch head only.
This is done by specifying ref: ${{ github.event.pull_request.head.ref }}
The event.pull_request.head.ref contains the branch name which is the reference to the head commit
The tests and builds themselves will be performed in the agents which will checkout the merged PR commit as usual.
Job - main
- uses: actions/checkout@v2
name: Checkout [main]
with:
fetch-depth: 0
Job - pr
- uses: actions/checkout@v2
with:
ref: ${{ github.event.pull_request.head.ref }}
fetch-depth: 0
Helping NX Affect Command Comparison
For PR's we want to compare the head of the PR branch against the base of the PR.
For main branch commits, the desire is to find changes against the latest commit that was succesful.
NX provide an action utility that calculate those head and base commits and set those to environment variables - NX_BASE and NX_HEAD - used by the NX Affected command.
- name: Derive appropriate SHAs for base and head for `nx affected` commands
uses: nrwl/nx-set-shas@v2
Starting the Distributed Execution Manager
To start the manager daemon, we execute nx-cloud utility with start-ci-run command.
- run: pnpm exec nx-cloud start-ci-run
Dispatching a Distributed Task
The NX Affect command run without customization.
The parallel/max-parallel does not specify the number of workers/agents but specify the number of processes to run parallel on each agent and on the manager.
- run: pnpm exec nx affected --target=build --parallel --max-parallel=3
Cleanup Agent After Success and Failure
After the job is done or after a build error, we want to signal the agents to shutdown and not keep waiting for future tasks to execute.
This is done by activating nx-cloud stop-all-agents.
In order to do this step even on error, the step needs to specify that it will run always and not just on success.
if: ${{ always() }} - Do the step always
- name: Clean Agents
if: ${{ always() }}
run: pnpm exec nx-cloud stop-all-agents
Agent Job Configuration
The Agent has one shared configuration. To parallel and multiple the number of agents node, we use the matrix feature of the job.
It will create parallel worker nodes, each with different agent name variable value.
strategy:
matrix:
agent: [1, 2]
Rest of the agent job configuration is standard. Checking out the repo merged commit, setting up nodejs, and starting up the agent server that listen for the tasks manager task dispatches.
- name: Start Nx Agent ${{ matrix.agent }}
run: npx nx-cloud start-agent
Conclusion
For monorepo scalable testing and building, checkout NX and NX Cloud
It provide an interesting solution for efficient parallel task execution to speedup monorepos CI workflows.
Top comments (0)