Create an auto-merging workflow on Github

Build servers have been the norm in software development for decades, with automated verifications of every change committed. If a developer submitted a bad change, they would be notified immediately.

But the breaking change is still on the main branch, possibly preventing team members from making progress. It would be less disruptive if the bad commit never reaches the main branch.

Submitting pull requests can help prevent this; and it is a popular way for teams to work. But pull requests are intended to handle the process for feedback of proposed changes. For single-developer repos, or teams that don't need this feedback process, using pull requests just to prevent broken builds from entering the default branch easily overcomplicates things.

A much simpler process is to just push to a different branch, have a build execute on the branch itself, and merge it automatically if the build passes.

This article shows how to setup such a workflow for github projects almost seamlessly, and how to deal with a few challenges along the way.

Set the permissions

This setup requires a github workflow that pushes to your repository. In order for this to work, you need to set the proper permissions for the project.

Go to the settings page for your project, and open the "Actions > General" settings, where you must configure "Workflow permissions" to "Read and write permissions".

Once this is done, workflow actions can push to your repository.

Create/update a verification workflow

You must have a verification workflow. I assume that you already have an existing workflow, and that it has a push trigger.

Normally, the trigger is not set to run on all branches, so you need to add the branch name to use. Here it's called auto-merge. If you don't already have a push trigger, be sure to add it.

Remember the name of the workflow, here it's Build. It is needed in the next step.

# .github/workflows/build.yaml
name: Build

on:
  push:
    branches: [ "main", "auto-merge" ]

For a team setting, each developer could have their own branch, and you can use a wildcard to react to them all.

About github workflows

If you are new to workflows, here are some fundamentals.

A github workflow is started from a trigger. A common configuration is to use both the push trigger, to run the workflow when there has been pushed to a branch, and the pull_request trigger, which obviously triggers when a pull request is created, or new code pushed to the pull request.

There are many kinds of triggers, including triggers that are completely unrelated to code. E.g., you could setup a scheduled job to renew server certificates.

A verification workflow would normally use actions/checkout to fetch the code from git. For a push trigger, the branch will by default be checked out. For a pull_request trigger, a merge commit will be checked out, i.e. it is the result of a merge with the default branch that is verified by the workflow.

A common default default is to have both push and pull_request triggers on the default branch, here main.

# .github/workflows/build.yaml
name: Build

on:
  push:
    branches: [ "main", "auto-merge" ]
  pull_request:
    branches: [ "main" ]


jobs:
  build:
    name: Build and test the code
    runs-on: ubuntu_latest
    steps:
    # Uses can use pre-made actions. 
    # actions/checkout will fetch your code.
    - uses: actions/checkout@v4
    # Frameworks can often be configured using other actions.
    # Github have good starting points for most project types.
    - name: Build and test
      # Make sure you run the steps necessary. 
      # Github have good starting points for most project types.
      run: ./build-and-test.sh

Create a new workflow in the default branch.

The new workflow uses the workflow_run trigger, a trigger that can react to events of another workflow.

Because this workflow is triggered by another workflow, not a branch or a pull request, it is global to the github project and must exist in the default branch; typically named main or master.

The auto-merge workflow should be run when the verification workflow is completed, and the workflow was executed on the auto-merge branch.

# .github/workflows/auto-merge.yaml
name: Auto-merge
on:
  workflow_run:
    workflows: [Build]
    branches: [auto-merge]
    types: [completed]

So while the workflow with a workflow_run trigger works on a project level, it can still filter on the branches that were the trigger a verification workflow run.

Note: You can use wildcards in your branch names if you have multiple auto-merge branches.

Create a job

A job does the actual work. While this is triggered on a completed verification workflow, we don't want to run the job on a failed verification. To handle that, a condition is added to check the outcome of the completed workflow.

jobs:
  on-success:
    runs-on: ubuntu-latest
    if: ${{ github.event.workflow_run.conclusion == 'success' }}

Fetch the code

To fetch the code, we use the actions/checkout action.

The first unexpected issue is that the action checks out the default branch, not the auto-merge branch; which was the original source of a trigger. While a push trigger will by default check out the branch that was pushed to, the workflow_run trigger does not. We must check out the right branch in the workflow ourselves.

The trigger has an associated event which contains information about the completed verification workflow, including the name of the branch that triggered the first workflow. This value is found in the variable github.event.workflow_run.head_branch.

Note: For a single auto-merging branch, we could just have duplicated the branch name, but for multiple branches, it's necessary to read this value from the event.

The next issue is that by default, the action creates a shallow clone, i.e., there is no history. You cannot push to a branch, if you don't locally have all history since the head of the target branch.

The easiest solution is to add fetch-depth: 0 to the action.

    steps:
      - uses: actions/checkout@v4
        with:
          ref: ${{ github.event.workflow_run.head_branch }}
          fetch-depth: 0

For large repositories with a lot of history, or large binary files in history, this can increase the runtime significantly. But there are ways to deal with this problem.

Smarter handling of shallow clones

Instead of fetching full history, we can fetch just enough history to be able to push.

Note: This is much more tricky, so for smaller repositories, the previous method would be advisable. Particularly, if you/the team don't feel comfortable with shell scripting.

The changes to make compared to the previous simple version are:

1. Remove the ref and fetch-depth options.
2. Use git remote set-branches ... to tell git there is a different remote branch we want to use.
3. Use git fetch ... to fetch the relevant commits, i.e. just enough shallow history, to be able to push the changes.
4. Checkout the source branch locally.

    steps:
      - uses: actions/checkout@v4
      - name: Tell git we want the `auto-merge` branch
        run: git remote set-branches --add origin ${{ github.event.workflow_run.head_branch }}
      - name: Fetch the target branch
        run: git fetch --shallow-since="`git show --no-patch --format=%ci HEAD`"
      - name: Checkout target branch
        run: git checkout ${{ github.event.workflow_run.head_branch }}

1. Remove ref and fetch-depth

Because we need to have all commits in the source branch not reachable from the target branch (default branch), we need the target branch in our working copy. The simplest way is to start with that branch checked out.

2. Add new remote branch

In a shallow clone, git doesn't fetch all remote branches. The following command tells git we specifically want to have the source branch.

git remote set-branches --add origin ${{ ... .head_branch }}

Again, for a single branch, you could duplicate the branch name.

3. Fetch the remote branch.

With the remote branch added, we can fetch the branch using git fetch.

In a shallow clone, fetch will fetch commits that are after the current branch, i.e., it will fetch all the commits that are new in the auto-merge branch. But if the target branch is ahead of the source branch, i.e., the default branch contains commits not yet in the auto-merge branch, git fetch fetches the entire history, defeating the purpose of the shallow clone to begin with.

This scenario is less likely for a single-developer setup, but very likely to happen occasionally in a team setup where team members use individual auto-merge branches.

To keep the clone shallow, the command option --shallow-since="" is used to not fetch commits older than the current HEAD (which is the default branch). The commit timestamp for HEAD is found using git show --no-patch --format=%ci HEAD.

git fetch --shallow-since="`git show --no-patch --format=%ci HEAD`"

Backticks here is a special shell feature that executes the git show command, and uses the text output in the command line for the git fetch command

Now the fetch command will fetch exactly all the commits that are necessary for the branch to be pushed if possible.

4. Check out the source branch

Now, there is a local shallow clone with just enough commits to be able to push to the remote.

Push

Both the simple, and the shallow clone workflow variations, have left us with the source branch
checked out. All that is left is to push it to the remote target branch, i.e. push auto-merge to main in this example. Here HEAD is used, so the script doesn't need to know what the branch actually is.

jobs:
  on-success:
    # ...
    steps:
      # ...
      - name: Push to main
        run: git push origin HEAD:main

By default, git will only perform a fast-forward merge on push. So the workflow will fail if there are new commits on the target branch. In this case, you need to deal with the conflict, by either merging or rebasing your changes off the new master; just as you normally would.

The full workflow file

This is the full workflow file. Adapt the following to your own workflow.

• The verification workflow is named Build.
• The branch to run on is named auto-merge.
• The default branch is named main.

# .github/workflows/auto-merge.yaml
name: Auto-merge
on:
  workflow_run:
    workflows: [Build]
    branches: [auto-merge]
    types: [completed]

jobs:
  on-success:
    runs-on: ubuntu-latest
    if: ${{ github.event.workflow_run.conclusion == 'success' }}
    steps:
      - uses: actions/checkout@v4
      - name: Tell git we want the `auto-merge` branch
        run: git remote set-branches --add origin ${{ github.event.workflow_run.head_branch }}
      - name: Fetch the target branch
        run: git fetch --shallow-since="`git show --no-patch --format=%ci HEAD`"
      - name: Checkout target branch
        run: git checkout ${{ github.event.workflow_run.head_branch }}
      - name: Push current head to main
        run: git push -v origin HEAD:main

Pushing to the right branch locally

You can push directly to the branch from the command line:

> git push origin HEAD:auto-merge

Here, the remote is assumed to be named origin, the default for most workflows. But writing this manually every time becomes troublesome. You can easily fix this in your local configuration, found in the .git/config file of your working directory.

By adding a push refspec, you can tell to push to a different remote branch.

# .git/config
[remote "origin"]
    url = git@github.com:username/repository.git
    fetch = +refs/heads/*:refs/remotes/origin/*
    push = refs/heads/main:refs/heads/auto-merge

With this setting, when you pull the main branch from your local working directory, you get the main branch from the remote repository, but when you push the main branch, it will be pushed to the auto-merge branch on the remote. If your changes are good, your team mates will quickly get them when they pull.

Note: This setting is not part of the git repository itself, it is only stored in the local git configuration.

Using this setup, you local workflow is exactly as if you were working directly on the default branch, but does change the defaults for all other branches.

A less intrusive configuration

A solution to not break default behaviour for other branches involves creating a new remote in the git configuration with the same url. We can specify that the default branch uses this remote when pushing, and now other branches are not affected by the change to the push configuration.

# .git/config
[remote "origin"]
    url = git@github.com:username/repository.git
    fetch = +refs/heads/*:refs/remotes/origin/*
[remote "origin-main"]
    url = git@github.com:username/repository.git
    fetch = +refs/heads/*:refs/remotes/origin/*
    push = refs/heads/main:refs/remotes/origin/auto-merge
[branch "main"]
    remote = origin
    pushRemote = origin-main
    merge = refs/heads/main

The local workflow

With everything setup, your workflow should looks remarkably familiar:

> git pull # or git pull --rebase
> git commit -am "Change 1"
> git commit -am "Change 2"
> git commit -am "Change 3"
> git push
# In case of a conflict
> git pull # or git pull --rebase
# Fix merge conflicts
> git push

The only differences are:

• There is the delay of the build before you know what the outcome is.
• You local branch will appear to be ahead of the default branch after a push, until you fetch after the build has succeeded.

But breaking changes will not appear on the default branch.

It's a simple solution to a simple problem

This setup will not prohibit bad commits from reaching the default branch. Developers can still push to that branch directly if they want.

For a team project, every developer needs configure their git configuration to push to an auto-merge branch, requiring uncommon initial custom configuration.

But for those projects that don't need complicated processes, this small change can help prevent bad commits from reaching the default branch in a completely non-intrusive way.