Preventing Git Rebase Fights

What’s a Git Rebase Fight?

Have you ever experienced this situation?

1. You go to merge your PR (pull-request), but the PR says it must be rebased before you can merge.

1. You rebase it, which kicks off a new build. But the build must complete before you’re allowed to merge.

1. The build completes! Yay! But while you were waiting, someone else managed to merge their PR, updating origin/master with their work. Uh oh!

1. You go back to step 1 and hope for better luck this time…

Bumping into this loop once or twice a month is not a big deal (especially if step 2 takes under 10 seconds). But sometimes the situation can become pathological.

Solution: Optimistic Build Status Propagation

You can use $(git diff TARGET...SOURCE | git patch-id) to prevent these rebase fights. This is handy when you know the build is very likely to succeed (e.g., squashes, amends, clean rebases, clean sync-merges).

The technique is called “Optimistic Build Status Propagation” because it uses the output of “git patch-id” as a heuristic to propagate build status to newer commit-ids without requiring the actual full build to finish. It works like this:

1. You push a new branch to your central git server.
2. The push triggers a build on your CI (continuous integration) server.
3. The build eventually succeeds. The branch’s tip commit is marked with a SUCCESS flag.
4. You decide to rebase your branch.
5. During the rebase your git server notices two things: 1. The commit before the rebase had a successful build filed against it, and 2. The rebase was clean (no conflicts).
6. The rebase triggers a build on your CI server. The CI server sends your git server an IN-PROGRESS notification.
7. Your git server receives the IN-PROGRESS notification. Because the git server also knows the rebase was clean, and also knows the pre-rebase commit had a SUCCESS flag, your git server optimistically marks the new tip commit with SUCCESS instead of IN-PROGRESS.

In other words, the “SUCCESS” flag from before the rebase is propagated to the commit created after the rebase. That’s why it’s called “Optimistic Build Status Propagation”. It lets you merge immediately after the rebase, since there’s no need to wait for the build to complete.

This optimistic window is temporary. Only the IN-PROGRESS flag is intercepted and replaced with a SUCCESS flag. Eventually the CI server will complete the rebased build and send a final SUCCESS or FAILURE notification. These are dutifully recorded of course, replacing any previous flags filed against the commit.

This technique is invaluable for shops running a fast-forward merge policy alongside a build-all-branches policy. If your builds are even just a little bit slow (e.g., 3 minutes or worse), your staff are probably waging an infinite rebase war against each other. Or they’ve found a better job somewhere else. Or they’ve disabled those merge policies.

If you’re on Bitbucket Server there is at least one plugin for this: PR-Booster for Bitbucket Server.

I’m not aware of any pre-baked solutions for this problem on Gitlab or Github.

What Causes Git Rebase Fights?

Fast-forward merge policy causes rebase fights.

A fast-forward merge policy only lets PRs merge if they are ahead of origin/master. In other words, PRs must be rebased before they can merge. The policy keeps git history neat, clean and linear by eliminating merge commits. But the policy can also cause rebase fights. I can think of two situations in particular where this happens:

1. High contention for the merge right-of-way and repo is so large that rebases are slow. I suspect this situation is rare and confined to very large teams. Rebases are only slow with very large repos, and you’d need at least 50+ engineers targeting the same origin/master before the contention would get high enough. Monorepos in particular may be vulnerable to this.
2. Slow secondary commit validation processes (e.g., builds must succeed before merge, but builds are 3+ minutes). I suspect this situation is much more common.

If your rebase fights are happening because of scenario 1 (very large repo + very large team), then you should probably forget about running a fast-forward policy. Sorry, but you need those merge commits. In exchange for a messier commit graph you get improved productivity. It’s a good tradeoff!

If your rebase fights are happening because of scenario 2 (slow secondary processes), then “Optimistic Build Status Propagation” is available as a solid mitigation. Under scenario 2 you can have both a clean commit graph and the productive team!

The rest of this blog post is about how $(git diff TARGET...SOURCE | git patch-id) works under the hood.

Triple-Dot-Diff and “git patch-id”

I refer to “git diff A…B” as triple-dot-diff. When people complain about Git’s usability, the triple-dot operator is certainly one of Git’s blemishes. The operator’s behaviour is inconsistent across various commands (e.g., “git log A…B” does something quite different).

The manual for “git diff” explains the triple-dot-diff like so:

“git diff A…B” is equivalent to “git diff $(git-merge-base A B) B”

Visually, it looks like this:

This is because the merge-base of master and branch is commit cc603d1, the last commit they had in common before they diverged. And so “git diff master…branch” is equivalent to “git diff cc603d1 ee7b565”.

Turns out clean rebases, squashes, merge-squashes, and sync-merges (and amends, of course) do not perturb this fundamental diff. The command “git merge…branch” (with three dots) is stable even if master advances or branch is rebased. The line numbers might change, and hunks might be rearranged, but the fundamental diff itself does not change unless there’s a conflict resolution (or an evil merge). Atlassian’s Auto Unapprove plugin explores this in detail in its issue #15.

If we were writing “Optimistic Build Status Propagation” from scratch, generating canonicalized diffs would be a big headache. Fortunately, the “git patch-id” command already has this covered, with some extra help coming from its “ — stable” option:

— stable

Use a “stable” sum of hashes as the patch ID. With this option

reordering file diffs that make up a patch does not affect the ID

Here’s some examples using master and branch from the diagram (clone from here if you must!):

git diff master...branch | git patch-id --stable
**790e0c0693c61e28fa1b3eea204bafe3946f5cba**

If I synch-merge (I’m on branch):

git merge master -m 'merge'
git diff master...branch | git patch-id --stable
**790e0c0693c61e28fa1b3eea204bafe3946f5cba**

If I retreat and rebase:

git reset --hard origin/branch
git rebase master
git diff master...branch | git patch-id --stable
**790e0c0693c61e28fa1b3eea204bafe3946f5cba**

The patch-id doesn’t change! This makes the command (triple-dot-diff piped into patch-id) perfect for determining when rebases and other common branch operations have not changed the underlying work sitting on the source branch. Since the underlying patch has not changed, one can optimistically presume the build will probably have the same result.

Conclusion

Fast-forward merges are great, because they avoid pointless merges and keep the history clean. Requiring successful builds before merging is great because it prevents broken builds. But add these together and you might find yourself in an infinite rebase fight!

Fortunately you can use $(git diff TARGET...SOURCE | git patch-id) to stop the fighting.

If you’re on Bitbucket Server, install the PR-Booster add-on to deploy the fix instantly.

Otherwise roll your own, and let me know when you do! Email me at julius at mergebase.com.

Happy rebasing!

(p.s. For those on Bitbucket Server, I use Control Freak to enforce a fast-forward merge policy on git repositories I control.)

This is a guest post for Bitbucket written by Julius Musseau at mergebase.com. To contribute to the Bitbucket blog, apply here.