Automatic versioning in a Lerna monorepo using Github actions

Prerequisites

This is a continuation of my previous article: Monorepo using Lerna, Conventional commits, and Github packages. The prerequisites of that previous article are assumed to understand this one, so you may want to read it first for extra context.

If you feel stuck at any point during the article or you simply want to get the code right now, it can be found in the repository.

Context

Continuous Integration (CI)

Continuous Integration is a practice in software development that consists in integrating the code as frequently as possible. Before integrating the code, it's common to execute a series of checks such as running tests or compiling/building the project, aiming for detecting errors the earlier the better.

A common practice is to automatically execute these checks when opening a new Pull Request or even pushing code to the repository to force that all these checks pass before we can safely integrate the changes into the codebase.

Github actions

Github actions are a Github feature that allows developers to execute workflows when certain events happen in our repositories, such as pushing code or closing a Pull Request(often used in Continuous Integration scenarios). Github actions are free of charge for public repositories.

These workflows are organized in jobs, steps and actions in a nested fashion, and are triggered by one or more events. Each workflow is a single file written in the YAML language.

What are we going to build?

We are going to automate the versioning and publication of the packages in our monorepo using Lerna (with conventional commits) and Github actions.

We are going to implement two different Github workflows:

1 - Checks workflow: When we open a new Pull Request or push changes to a pull request that is open, it will run a set of checks that we consider essential for integrating the changes into our codebase.

2 - Publish workflow: Whenever a Pull Request is merged, we'll execute a workflow that will version and publish our packages. It will behave slightly different depending on the destination branch:

• When merged against the development branch, it will publish beta versions of the changed packages (suitable for QA or testing).
• When merged against the main branch, it will publish final versions (ready for production).

We will start from an existing monorepo that already contains two javascript packages that I created for this previous article.

The following picture illustrates the workflows that we will implement in Github actions terminology:

Hands-on

Part 1 - Checks workflow on PR open/modified

Github expects workflows to be located under the ${projectFolder}/.github/workflows, so let's create a new Github branch and add our first workflow checks.yaml inside that directory (you can create workflows from the Github UI too):

The project structure looks like this:

/
  .github/
    workflows/
      checks.yaml
  [...]

Now, let's start working on the workflow. Open the checks.yaml file in an editor and add the following attributes:

name: Checks # Workflow name

on:
  pull_request:
    types: [opened, synchronize] # Workflow triggering events

• name: The name of the workflow.
• on: The listener of the event(s) that will trigger this workflow. In our case, it will be triggered every time that a Pull Request gets opened or modified.

Next, we will add a job to the workflow and configure the type of instance that Github will spin up for running it with the runs-on attribute:

name: Checks
on:
  pull_request:
    types: [opened, synchronize]

jobs: # A workflow can have multiple jobs
  checks: # Name of the job
    runs-on: ubuntu-latest # Instance type where job will run

This job will contain several steps:

• Checkout: Get the code from the repository where the workflow is defined.
• Setup NodeJS: Setup NodeJS with a specific version.
• Setup npm: Since we will install dependencies from our private registry (in Github packages), we have to add it to the npm config.
• Install dependencies: Install the needed npm packages.
• Run tests: Execute tests, if any.

In a real-world project it's likely that we run other steps such as checking syntax using a linter, building the project or running any other check/process that we consider essential to mark the changes as valid before integrating them into the codebase.

Custom vs public actions

For some of the mentioned steps we will write the commands from scratch but for others, we will take advantage of existing public actions that have been created by the community and are available in the Github marketplace.

The public actions use the uses keyword and the custom commands (single or multiple lines) use the run one.

Let's implement the first two steps of the build job:

name: Checks
on:
  pull_request:
    types: [opened, synchronize]
jobs:
  check:
    runs-on: ubuntu-latest

    steps:
    - name: "Checkout" # Download code from the repository
      uses: actions/checkout@v2 # Public action
      with:
        fetch-depth: 0 # Checkout all branches and tags

    - name: "Use NodeJS 14" # Setup node using version 14
      uses: actions/setup-node@v2 # Public action
      with: 
        node-version: '14'

• The Checkout step will download the code from the repository. We have to add the depth: 0 option so Lerna can properly track the tags of the published packages versions and propose new versions when it detects changes.

• In the Use NodeJS 14 step we are configuring NodeJS to use the version 14 but we could even execute it for multiple versions at once using a matrix.

Let's commit and push this version of the workflow to Github and open a Pull Request afterward (if you don't have a development branch already created, create one from main because we'll open the pull request against it).

Once the Pull Request has been opened our workflow will be executed. Open a browser and navigate to the "Actions" section of the repository to see the execution result:

If we click on it, we can see the execution details, and by clicking on any of the jobs (in our case, the checks job) we will be able to see the status and outputs of each of its steps:

Let's add the next step: Setup npm. In this step, we'll add our Github packages registry to the .npmrc file so npm can find the packages published in our Github packages registry.

One or multiple commands can be executed in every step action. In this case, we'll run a couple of npm set commands in the same action:

name: Checks
on:
  pull_request:
    types: [opened, synchronize]

jobs:
  checks:
    runs-on: ubuntu-latest

    steps:
    - name: "Checkout"
      uses: actions/checkout@v2
      with:
        fetch-depth: 0

    - name: "Use NodeJS 14"
      uses: actions/setup-node@v2
      with: 
        node-version: '14'

    - name: "Setup npm" # Add our registry to npm config
      run: | # Custom action
        npm set @xcanchal:registry=https://npm.pkg.github.com/xcanchal
        npm set "//npm.pkg.github.com/:_authToken=${{ secrets.GITHUB_TOKEN }}"

Workflow environment variables

In the previous snippet, you'll have noticed the secrets.GITHUB_TOKEN. This environment variable is added by Github and can be used to authenticate in our workflow when installing or publishing packages (know more).

A part from that one, Github adds other variables such as the branch name or the commit hash, which can be used for different purposes. The complete list is available here.

Next, we'll add another step: Install dependencies. In this step action we'll install the root dependencies in production mode (see npm ci command) as well as running lerna bootstrap for installing the dependencies for each of our packages and create links between them.

name: Checks
on:
  pull_request:
    types: [opened, synchronize]

jobs:
  checks:
    runs-on: ubuntu-latest

    steps:
    - name: "Checkout"
      uses: actions/checkout@v2
      with:
        fetch-depth: 0

    - name: "Use NodeJS 14"
      uses: actions/setup-node@v2
      with:
        node-version: '14'

    - name: "Setup npm"
      run: |
        npm set @xcanchal:registry=https://npm.pkg.github.com/xcanchal
        npm set "//npm.pkg.github.com/:_authToken=${{ secrets.GITHUB_TOKEN }}"

    - name: Install dependencies
      run: | # Install and link dependencies
        npm ci
        npx lerna bootstrap

Commit and push the changes and see how the "Pull Request synchronized" event triggers our workflow, which now contains the last steps that we added:

Before adding our last step Running tests we need to make a change in our date-logic and date-renderer packages, modifying the npm test script. Since we haven't implemented any actual test yet, we'll simple echo "TESTS PASSED" when that command is executed.

Modify the test script in the package.json of the date-logic package and push the changes to the repo. Then, repeat the same process for the date-renderer.

# package.json
"scripts": {
  "test": "echo TESTS PASSED"
}

# commit and push
$ git add .
$ git commit -m "feat(date-logic): echo tests"
$ git push

After pushing the new test command to our packages we can add the Running tests step to our workflow.

name: Checks
on:
  pull_request:
    types: [opened, synchronize]

jobs:
  checks:
    runs-on: ubuntu-latest

    steps:
    - name: "Checkout"
      uses: actions/checkout@v2
      with:
        fetch-depth: 0

    - name: "Use NodeJS 14"
      uses: actions/setup-node@v2
      with:
        node-version: '14'

    - name: "Setup npm"
      run: |
        npm set @xcanchal:registry=https://npm.pkg.github.com/xcanchal
        npm set "//npm.pkg.github.com/:_authToken=${{ secrets.GITHUB_TOKEN }}"

    - name: Install dependencies
      run: |
        npm ci
        npx lerna bootstrap

    - name: Run tests # Run tests of all packages
      run: npx lerna exec npm run test

Push the changes to the repository and see the execution results in the Github actions section:

Congrats! we completed our first job and half of this tutorial.

Part 2 - Publish workflow on PR merged

Create a publish.yaml file under the workflows repository with the following content. You'll notice that we added a new branches attribute to the event listeners. With this configuration, we're telling Github that only executes this workflow when a Pull Request is merged either against development or main branch.

name: Publish

on:
  pull_request:
    types: [closed]
    branches:
      - development
      - main

Now, we'll add a job named publish to this workflow, the runs-on attribute and a new one that we haven't used yet: if. This attribute is used to evaluate an expression to conditionally trigger the job if it evaluates to true or false (it can be used in steps too).

According to the on attribute that we configured, this workflow will trigger on every "Pull Request closed" event against development or main, but what we actually want is to execute it ONLY when the Pull Request has been merged (not discarded). Therefore, we have to add the github.event.pull_request.merged == true condition to the job:

name: Publish
on:
  pull_request:
    types: [closed]
    branches:
      - development
      - main

jobs:
  publish:
    if: github.event.pull_request.merged == true # Condition
    runs-on: ubuntu-latest

Now, let's replicate the same first three steps that we added in the checks workflow (Checkout, Use NodeJS 14 and Setup npm)

name: Publish

on:
  pull_request:
    types: [closed]
    branches:
      - development
      - main

jobs:
  publish:
    if: github.event.pull_request.merged == true
    runs-on: ubuntu-latest

    steps:
    - name: "Checkout"
      uses: actions/checkout@v2
      with:
        fetch-depth: 0

    - name: "Use NodeJS 14"
      uses: actions/setup-node@v2
      with:
        node-version: '14'

    - name: "Setup npm"
      run: |
        npm set @xcanchal:registry=https://npm.pkg.github.com/xcanchal
        npm set "//npm.pkg.github.com/:_authToken=${{ secrets.GITHUB_TOKEN }}"

Finally, we will add the final (and interesting) step: Publish and version. Let's analyze in detail the step attributes and the commands inside the action:

• Since Lerna will be in charge of publishing new versions of the packages, we have to set the GH_TOKEN environment variable with our Personal Access Token as the value, so Lerna has the required permissions.
• We have to add a couple of Github configuration lines to specify the username and email credentials, so Lerna can make commits and create tags for the new versions in the repository. For that, we'll take advantage of the github.actor variable available in the environment.
• In the if/else statement we're checking the ${{ github.base_ref }} variable to see if the destination branch of the PR is development. In that case, we will send the --conventional-prerelease and the --preid flags to the Lerna version command to generate beta versions. Otherwise (it only can be main because we restricted at the workflow level that it must be one of these two branches), we will use the --conventional-graduate argument to generate final versions. Last but not least, the --yes flag autoconfirms the version and publish operations (otherwise Lerna would prompt for manual confirmation and the CI would fail).

name: Publish
on:
  pull_request:
    types: [closed]
    branches:
      - development
      - main

jobs:
  publish:
    if: github.event.pull_request.merged == true
    runs-on: ubuntu-latest

    steps:
    - name: "Checkout"
      uses: actions/checkout@v2
      with:
        fetch-depth: 0

    - name: "Use NodeJS 14"
      uses: actions/setup-node@v2
      with:
        node-version: '14'

    - name: "Version and publish" # Interesting step
      env:
        GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
      run: |
        git config user.name "${{ github.actor }}"
        git config user.email "${{ github.actor}}@users.noreply.github.com"

        if [ ${{ github.base_ref }} = development ]; then
          npx lerna version --conventional-commits --conventional-prerelease --preid beta --yes
        else
          npx lerna version --conventional-commits --conventional-graduate --yes
        fi

        npx lerna publish from-git --yes

Let's commit the new workflow to the repository and merge the Pull Request afterward, so it gets triggered. If we inspect the output of the Version and publish step we can see a lot of information about the two steps that Lerna executed:

1) When running the lerna version command, it detected the changes in the packages and proposed new beta versions (notice the -beta.1 prefix) that were auto-accepted. After that, it pushed the version tags to the Github repo:

2) When running the lerna publish from-git command, it analyzed the latest Github tags to determine the versions that had to be published and published the packages to the Github package registry.

So now we have some testing versions in our Github packages registry:

We'll assume that they have been through testing and that are marked as ready for production. Let's create a new Pull Request from development against master, merge it and see how the same Publish job is executed, but this time Lerna will publish the final versions:

Conclusion

We have seen how powerful a couple of Lerna commands can be (in conjunction with a proper conventional commits history) for the Continuous Integration workflows of our monorepos.

By automating these steps we can forget about having to manually decide the versions for all of our packages and thus, avoiding human errors. In this case, we used Github actions for doing it but any other tool such as Jenkins or CircleCI would work too.

Next steps

• Configure Github branch protection rules to block the Pull Request merge button if the checks workflow failed.
• Set up a commit syntax checker (e.g. commitlint) to avoid human mistakes that could impact the versioning due to an inconsistent commit history.

Follow me on Twitter for more content @xcanchal

Buy me a coffee: