DEV Community

Cover image for Automatic versioning in a Lerna monorepo using Github actions
Xavier Canchal
Xavier Canchal

Posted on • Updated on

Automatic versioning in a Lerna monorepo using Github actions

Prerequisites

This is a continuation of my previous article: Monorepo using Lerna, Conventional commits, and Github packages. The prerequisites of that previous article are assumed to understand this one, so you may want to read it first for extra context.

If you feel stuck at any point during the article or you simply want to get the code right now, it can be found in the repository.

Context

Continuous Integration (CI)

Continuous Integration is a practice in software development that consists in integrating the code as frequently as possible. Before integrating the code, it's common to execute a series of checks such as running tests or compiling/building the project, aiming for detecting errors the earlier the better.

A common practice is to automatically execute these checks when opening a new Pull Request or even pushing code to the repository to force that all these checks pass before we can safely integrate the changes into the codebase.

Github actions

Github actions are a Github feature that allows developers to execute workflows when certain events happen in our repositories, such as pushing code or closing a Pull Request(often used in Continuous Integration scenarios). Github actions are free of charge for public repositories.

These workflows are organized in jobs, steps and actions in a nested fashion, and are triggered by one or more events. Each workflow is a single file written in the YAML language.

What are we going to build?

We are going to automate the versioning and publication of the packages in our monorepo using Lerna (with conventional commits) and Github actions.

We are going to implement two different Github workflows:

1 - Checks workflow: When we open a new Pull Request or push changes to a pull request that is open, it will run a set of checks that we consider essential for integrating the changes into our codebase.

2 - Publish workflow: Whenever a Pull Request is merged, we'll execute a workflow that will version and publish our packages. It will behave slightly different depending on the destination branch:

  • When merged against the development branch, it will publish beta versions of the changed packages (suitable for QA or testing).
  • When merged against the main branch, it will publish final versions (ready for production).

We will start from an existing monorepo that already contains two javascript packages that I created for this previous article.

The following picture illustrates the workflows that we will implement in Github actions terminology:

Workflows illustrated

Hands-on

Part 1 - Checks workflow on PR open/modified

Github expects workflows to be located under the ${projectFolder}/.github/workflows, so let's create a new Github branch and add our first workflow checks.yaml inside that directory (you can create workflows from the Github UI too):

The project structure looks like this:

/
  .github/
    workflows/
      checks.yaml
  [...]
Enter fullscreen mode Exit fullscreen mode

Now, let's start working on the workflow. Open the checks.yaml file in an editor and add the following attributes:

name: Checks # Workflow name

on:
  pull_request:
    types: [opened, synchronize] # Workflow triggering events
Enter fullscreen mode Exit fullscreen mode
  • name: The name of the workflow.
  • on: The listener of the event(s) that will trigger this workflow. In our case, it will be triggered every time that a Pull Request gets opened or modified.

Next, we will add a job to the workflow and configure the type of instance that Github will spin up for running it with the runs-on attribute:

name: Checks
on:
  pull_request:
    types: [opened, synchronize]

jobs: # A workflow can have multiple jobs
  checks: # Name of the job
    runs-on: ubuntu-latest # Instance type where job will run
Enter fullscreen mode Exit fullscreen mode

This job will contain several steps:

  • Checkout: Get the code from the repository where the workflow is defined.
  • Setup NodeJS: Setup NodeJS with a specific version.
  • Setup npm: Since we will install dependencies from our private registry (in Github packages), we have to add it to the npm config.
  • Install dependencies: Install the needed npm packages.
  • Run tests: Execute tests, if any.

In a real-world project it's likely that we run other steps such as checking syntax using a linter, building the project or running any other check/process that we consider essential to mark the changes as valid before integrating them into the codebase.

Custom vs public actions

For some of the mentioned steps we will write the commands from scratch but for others, we will take advantage of existing public actions that have been created by the community and are available in the Github marketplace.

The public actions use the uses keyword and the custom commands (single or multiple lines) use the run one.

Let's implement the first two steps of the build job:

name: Checks
on:
  pull_request:
    types: [opened, synchronize]
jobs:
  check:
    runs-on: ubuntu-latest

    steps:
    - name: "Checkout" # Download code from the repository
      uses: actions/checkout@v2 # Public action
      with:
        fetch-depth: 0 # Checkout all branches and tags

    - name: "Use NodeJS 14" # Setup node using version 14
      uses: actions/setup-node@v2 # Public action
      with: 
        node-version: '14'
Enter fullscreen mode Exit fullscreen mode
  • The Checkout step will download the code from the repository. We have to add the depth: 0 option so Lerna can properly track the tags of the published packages versions and propose new versions when it detects changes.

  • In the Use NodeJS 14 step we are configuring NodeJS to use the version 14 but we could even execute it for multiple versions at once using a matrix.

Let's commit and push this version of the workflow to Github and open a Pull Request afterward (if you don't have a development branch already created, create one from main because we'll open the pull request against it).

Once the Pull Request has been opened our workflow will be executed. Open a browser and navigate to the "Actions" section of the repository to see the execution result:

Action list

If we click on it, we can see the execution details, and by clicking on any of the jobs (in our case, the checks job) we will be able to see the status and outputs of each of its steps:

Job steps detail

Let's add the next step: Setup npm. In this step, we'll add our Github packages registry to the .npmrc file so npm can find the packages published in our Github packages registry.

One or multiple commands can be executed in every step action. In this case, we'll run a couple of npm set commands in the same action:

name: Checks
on:
  pull_request:
    types: [opened, synchronize]

jobs:
  checks:
    runs-on: ubuntu-latest

    steps:
    - name: "Checkout"
      uses: actions/checkout@v2
      with:
        fetch-depth: 0

    - name: "Use NodeJS 14"
      uses: actions/setup-node@v2
      with: 
        node-version: '14'

    - name: "Setup npm" # Add our registry to npm config
      run: | # Custom action
        npm set @xcanchal:registry=https://npm.pkg.github.com/xcanchal
        npm set "//npm.pkg.github.com/:_authToken=${{ secrets.GITHUB_TOKEN }}"
Enter fullscreen mode Exit fullscreen mode

Workflow environment variables

In the previous snippet, you'll have noticed the secrets.GITHUB_TOKEN. This environment variable is added by Github and can be used to authenticate in our workflow when installing or publishing packages (know more).

A part from that one, Github adds other variables such as the branch name or the commit hash, which can be used for different purposes. The complete list is available here.

Next, we'll add another step: Install dependencies. In this step action we'll install the root dependencies in production mode (see npm ci command) as well as running lerna bootstrap for installing the dependencies for each of our packages and create links between them.

name: Checks
on:
  pull_request:
    types: [opened, synchronize]

jobs:
  checks:
    runs-on: ubuntu-latest

    steps:
    - name: "Checkout"
      uses: actions/checkout@v2
      with:
        fetch-depth: 0

    - name: "Use NodeJS 14"
      uses: actions/setup-node@v2
      with:
        node-version: '14'

    - name: "Setup npm"
      run: |
        npm set @xcanchal:registry=https://npm.pkg.github.com/xcanchal
        npm set "//npm.pkg.github.com/:_authToken=${{ secrets.GITHUB_TOKEN }}"

    - name: Install dependencies
      run: | # Install and link dependencies
        npm ci
        npx lerna bootstrap
Enter fullscreen mode Exit fullscreen mode

Commit and push the changes and see how the "Pull Request synchronized" event triggers our workflow, which now contains the last steps that we added:

Screenshot 2021-10-03 at 22.42.41

Before adding our last step Running tests we need to make a change in our date-logic and date-renderer packages, modifying the npm test script. Since we haven't implemented any actual test yet, we'll simple echo "TESTS PASSED" when that command is executed.

Modify the test script in the package.json of the date-logic package and push the changes to the repo. Then, repeat the same process for the date-renderer.

# package.json
"scripts": {
  "test": "echo TESTS PASSED"
}
Enter fullscreen mode Exit fullscreen mode
# commit and push
$ git add .
$ git commit -m "feat(date-logic): echo tests"
$ git push
Enter fullscreen mode Exit fullscreen mode

After pushing the new test command to our packages we can add the Running tests step to our workflow.

name: Checks
on:
  pull_request:
    types: [opened, synchronize]

jobs:
  checks:
    runs-on: ubuntu-latest

    steps:
    - name: "Checkout"
      uses: actions/checkout@v2
      with:
        fetch-depth: 0

    - name: "Use NodeJS 14"
      uses: actions/setup-node@v2
      with:
        node-version: '14'

    - name: "Setup npm"
      run: |
        npm set @xcanchal:registry=https://npm.pkg.github.com/xcanchal
        npm set "//npm.pkg.github.com/:_authToken=${{ secrets.GITHUB_TOKEN }}"

    - name: Install dependencies
      run: |
        npm ci
        npx lerna bootstrap

    - name: Run tests # Run tests of all packages
      run: npx lerna exec npm run test
Enter fullscreen mode Exit fullscreen mode

Push the changes to the repository and see the execution results in the Github actions section:

Tests passed

Congrats! we completed our first job and half of this tutorial.

Part 2 - Publish workflow on PR merged

Create a publish.yaml file under the workflows repository with the following content. You'll notice that we added a new branches attribute to the event listeners. With this configuration, we're telling Github that only executes this workflow when a Pull Request is merged either against development or main branch.

name: Publish

on:
  pull_request:
    types: [closed]
    branches:
      - development
      - main
Enter fullscreen mode Exit fullscreen mode

Now, we'll add a job named publish to this workflow, the runs-on attribute and a new one that we haven't used yet: if. This attribute is used to evaluate an expression to conditionally trigger the job if it evaluates to true or false (it can be used in steps too).

According to the on attribute that we configured, this workflow will trigger on every "Pull Request closed" event against development or main, but what we actually want is to execute it ONLY when the Pull Request has been merged (not discarded). Therefore, we have to add the github.event.pull_request.merged == true condition to the job:

name: Publish
on:
  pull_request:
    types: [closed]
    branches:
      - development
      - main

jobs:
  publish:
    if: github.event.pull_request.merged == true # Condition
    runs-on: ubuntu-latest
Enter fullscreen mode Exit fullscreen mode

Now, let's replicate the same first three steps that we added in the checks workflow (Checkout, Use NodeJS 14 and Setup npm)

name: Publish

on:
  pull_request:
    types: [closed]
    branches:
      - development
      - main

jobs:
  publish:
    if: github.event.pull_request.merged == true
    runs-on: ubuntu-latest

    steps:
    - name: "Checkout"
      uses: actions/checkout@v2
      with:
        fetch-depth: 0

    - name: "Use NodeJS 14"
      uses: actions/setup-node@v2
      with:
        node-version: '14'

    - name: "Setup npm"
      run: |
        npm set @xcanchal:registry=https://npm.pkg.github.com/xcanchal
        npm set "//npm.pkg.github.com/:_authToken=${{ secrets.GITHUB_TOKEN }}"
Enter fullscreen mode Exit fullscreen mode

Finally, we will add the final (and interesting) step: Publish and version. Let's analyze in detail the step attributes and the commands inside the action:

  • Since Lerna will be in charge of publishing new versions of the packages, we have to set the GH_TOKEN environment variable with our Personal Access Token as the value, so Lerna has the required permissions.
  • We have to add a couple of Github configuration lines to specify the username and email credentials, so Lerna can make commits and create tags for the new versions in the repository. For that, we'll take advantage of the github.actor variable available in the environment.
  • In the if/else statement we're checking the ${{ github.base_ref }} variable to see if the destination branch of the PR is development. In that case, we will send the --conventional-prerelease and the --preid flags to the Lerna version command to generate beta versions. Otherwise (it only can be main because we restricted at the workflow level that it must be one of these two branches), we will use the --conventional-graduate argument to generate final versions. Last but not least, the --yes flag autoconfirms the version and publish operations (otherwise Lerna would prompt for manual confirmation and the CI would fail).
name: Publish
on:
  pull_request:
    types: [closed]
    branches:
      - development
      - main

jobs:
  publish:
    if: github.event.pull_request.merged == true
    runs-on: ubuntu-latest

    steps:
    - name: "Checkout"
      uses: actions/checkout@v2
      with:
        fetch-depth: 0

    - name: "Use NodeJS 14"
      uses: actions/setup-node@v2
      with:
        node-version: '14'

    - name: "Version and publish" # Interesting step
      env:
        GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
      run: |
        git config user.name "${{ github.actor }}"
        git config user.email "${{ github.actor}}@users.noreply.github.com"

        if [ ${{ github.base_ref }} = development ]; then
          npx lerna version --conventional-commits --conventional-prerelease --preid beta --yes
        else
          npx lerna version --conventional-commits --conventional-graduate --yes
        fi

        npx lerna publish from-git --yes
Enter fullscreen mode Exit fullscreen mode

Let's commit the new workflow to the repository and merge the Pull Request afterward, so it gets triggered. If we inspect the output of the Version and publish step we can see a lot of information about the two steps that Lerna executed:

1) When running the lerna version command, it detected the changes in the packages and proposed new beta versions (notice the -beta.1 prefix) that were auto-accepted. After that, it pushed the version tags to the Github repo:

Proposed versions and tags

2) When running the lerna publish from-git command, it analyzed the latest Github tags to determine the versions that had to be published and published the packages to the Github package registry.

Found packages to publish

Published date-renderer

Published date-logic

So now we have some testing versions in our Github packages registry:

date-logic beta

date-renderer beta

We'll assume that they have been through testing and that are marked as ready for production. Let's create a new Pull Request from development against master, merge it and see how the same Publish job is executed, but this time Lerna will publish the final versions:

Proposed final versions

Final date-renderer version

Final date-logic version

Conclusion

We have seen how powerful a couple of Lerna commands can be (in conjunction with a proper conventional commits history) for the Continuous Integration workflows of our monorepos.

By automating these steps we can forget about having to manually decide the versions for all of our packages and thus, avoiding human errors. In this case, we used Github actions for doing it but any other tool such as Jenkins or CircleCI would work too.

Next steps

  • Configure Github branch protection rules to block the Pull Request merge button if the checks workflow failed.
  • Set up a commit syntax checker (e.g. commitlint) to avoid human mistakes that could impact the versioning due to an inconsistent commit history.

Follow me on Twitter for more content @xcanchal

Buy me a coffee:

Image description

Top comments (15)

Collapse
 
benyitzhaki profile image
Ben Yitzhaki

thanks, great post! One comment though, I would look into replacing the custom token with GITHUB_TOKEN to follow best practices security wise. according to their docs "GitHub Packages allows you to push and pull packages through the GITHUB_TOKEN available to a GitHub Actions workflow."

docs.github.com/en/packages/managi...

Collapse
 
xcanchal profile image
Xavier Canchal

Great suggestion Ben. I was doubtful about wether the GITHUB_TOKEN would allow authenticating against package registries and I see it does!

Collapse
 
srd profile image
subhranshu das

What an awesome series, clarified lot of Lerna sticking points for me.
Questions-
1) lets say you keep add 3 merges to development without pulling to main
so how does the versions work out?
a) after 1st merge (fix) in development 2.1.0-beta.0 --> 2.1.0-beta.1 ? or 2.1.1-beta.0?
b) assuming 2.1.1-beta.0, then 2nd merge (fix) in development 2.1.2-beta.0
c) then 3rd merge (fix) in development 2.1.3-beta.0
Now when we merge to main then it becomes 2.1.3? hence directly jumping from
2.1.0 to 2.1.3 in main?
Any good reading resources on semantic relases apart from the dry semVer doc??

2) Do you see if there is a need to changes fetch depth in the YAML - github.com/lerna/lerna/issues/2542

Collapse
 
sfilinsky profile image
SFilinsky

Thanks for post, great help! Question for me is how do you upgrade project version? After you merge to master, it removes -beta postfix, but how do you upgrade package version from 2.1.0 further to 2.1.0-beta.0? And how can it determine if to change major, minor or patch number?

Collapse
 
xcanchal profile image
Xavier Canchal

Hi SFilisnky. Lerna parses the conventional commit history, which include of the scope change, to determine the major, minor or patch version (see conventional commits specification). For the -beta removal, it’s the other way around. Let’s say you are in 2.1.0 and you merge a patch PR against development. Then, Lerna will bump to 2.1.1-beta.0 and when merging against master it will remove the suffix, leaving the final 2.1.1 version. Does that make sense?

Collapse
 
srd profile image
subhranshu das

Had a query on the usage of the packages after they are published-
let's say
step1) changes merged to dev --> publishes --> 1.0.1-beta.0
step2) the changes from dev are now merged to main --> publishes --> 1.0.1

Now can an end user install both 1.0.1-beta.0 & 1.0.1 ?

Collapse
 
xcanchal profile image
Xavier Canchal

Yes, they are published and available in the registry unless you delete them.

Collapse
 
srd profile image
subhranshu das

How can we add conventional commits for a git commit message which involves ay changes in 3 packages - pkg1, pkg2, pkg3 ?
Scenario
Previously,
pkg1 - 1.0.0
pkg1 - 1.0.0
pkg1 - 1.0.0

git commit -m "
feat(pkg1): added button color
fix(pkg2): fixed typo
fix(pkg3): fixed header
"

Now,
pkg1 - 1.1.0
pkg1 - 1.0.1
pkg1 - 1.0.1

Is that so? Also how to add breaking changes in the multiple package change commit where 1 of the packages have a breaking change, others do not.

Thanks!!!

Collapse
 
xcanchal profile image
Xavier Canchal

Hi! I would commit the changes of each package separately (one commit for each package)

Collapse
 
richfrost profile image
Rich Frost

Is this still a workable solution? I followed your steps but on the version/publish step I get the error:
EUNCOMMIT Working tree has uncommitted changes, please commit or remove the following changes before continuing:

because the version command has updated the package.json and is therefore not able to carry on.

Collapse
 
alex_boulay_036612da9a348 profile image
Alex Boulay • Edited

Thank you for the great post! Got everything working except one thing that is bugging me:

When using --conventional-prerelease, lerna creates the correct changelogs(with the correct commit history)

but when I use --conventional-graduate, lerna only puts this into my changelog:
Note: Version bump only for package

Also this only happens when running the command in github action, outside of it, it's fine!

Do you know why? Thanks!

Collapse
 
tushar199 profile image
Tushar Mistry

Hi Some Edge Cases I wanted to discuss
So we have master and dev branch merging to dev branch npm ci works perfectly and versioning is performed and beta is applied. But now if you try to merge this branch into master your lockfile is outdated npm ci would throw error cant perform installing node_modules.

PS: using pnpm as package manager

Collapse
 
smerth profile image
Stephen Merth

This is a great post, thanks. When I create a new Pull Request from development against master (main), in my case, I end up with merge conflict. The issue is that the development branch has beta versions and the main has major versions. So there is a conflict everywhere the version is defined. This can't be the desired behavior but I can't see a way around it... Maybe I misunderstand something about the workflow? Any Ideas about where I am going wrong?

Collapse
 
danicaliforrnia profile image
Daniel Stefanelli

have you found a better branching approach to solve that problem?

Collapse
 
tutods profile image
Daniel Sousa @TutoDS

Hi.
for me the packages doesn't appear on the repository, and other thing, how can I list the same packages on npmjs.com?