You have messed up production. All hell broke loose. What to do now? Fix it as fast as possible and undo the last change that made everything fall apart to unblock further deployments.
Fix Production Fast
First of all, it is a good idea to get back to a working version of the application as fast as possible. Many hosting platforms have some sort of rollback functionality (like Vercel's instant rollback) that you can use to get back to a working application in seconds. Or you deploy the last working commit from your local machine (be careful to not make things worse; you can leverage git bisect to find the last working commit). This is only a temporary solution. The HEAD of your main branch will still point to the broken application version. So how to fix that?
Undo Your Fuckup – Git Revert
This section is about getting your main branch (or trunk or whatever you want to call it) back to a functional state to unblock further deployments (and to prevent yourself from accidentally deploying the broken application again using your CI/CD pipeline).
Remove Last Commit (not git revert
)
You could just remove your last commit using git reset HEAD^ --hard
and git push --force
and get rid of that commit (assuming that this has introduced the error). This is rarely a good idea. You don't want to rewrite the git history of a shared remote branch. (Even if you are alone in this project I'd prefer the next solution). It can introduce conflicts with your co-workers' local working version of that branch. But how to do better?
Here is where git revert
comes into place.
Git Revert (undo) the Last Commit
git revert
is used to record some new commits to reverse the effect of some earlier commits (often only a faulty one).
(Source: Git revert documentation)
This is what we want. We create a new commit (not changing the existing commits and messing with our co-workers' sanity) that applies the changes that the commit to be reverted introduced. We can revert the changes of the last commit by finding its SHA using git log
and run git revert SHA
:
Commit SHA3
now applies the changes that are required to get back from SHA2
to SHA1
. Problem solved and we are good to go, right? Well, in this simple case, yes. But often we work with Pull Requests (PRs) and create merge commits introducing our (shitty) changes to the main branch. Trying the above will result in this error:
git revert SHA2
# error: commit SHA2 is a merge but no -m option was given.
# fatal: revert failed
git revert
fails and you start to sweat. Let's explore what this error is about and how to fix it.
Git Revert a Merge Commit
The part "but no -m option was given" of the previous error is already hinting at the solution. What is the -m
option of git revert
?
-m parent-number
--mainline parent-number
Usually you cannot revert a merge because you do not know which side of the merge should be considered the mainline. This option specifies the parent number (starting from 1) of the mainline and allows revert to reverse the change relative to the specified parent.
(Source: Git revert documentation)
Okay, next question: What does "which side of the merge should be considered the mainline" mean?
Let's have a look at this scenario:
As you can see, SHA3
which introduced the issues has two parent branches (SHA1
and SHA2
). How should git know which one it should revert to (the mainline)? This is what you have to tell git revert
with the -m
option.
How to find the "parent number" that you have to pass to git revert -m
? You can use git catfile -p SHA3
for that or look at the commit message of the merge commit using git log
:
git log
tells you that SHA3
is a merge commit of SHA1
and SHA2
. git cat-file
shows you details for commit SHA3
and tells you that SHA3
has two parents (basically the same info as in the commit message). In both cases the first mentioned SHA is parent-number
1 (the second 2 etc.). In order to revert the changes introduced in commit SHA3
and get back to SHA1
you have to execute git revert -m 1 SHA3
. Now you are all set and can push the branch with the new commit.
Now you know to revert merge commits and understand what you are doing instead of copying the command from Stackoverflow over and over again 😜
Conclusion
It can become stressful if you are in the situation of having to fix a production problem that you (or a co-worker) have introduced. Things get really bad if you face a "fatal" error trying to revert a merge commit. Make sure you can deal with these situations before you have to deal with them.
The time to repair a roof is when the sun is shining.
- OR -
The time to learn how to deal with a production issue is when production is not broken.
Top comments (0)