Submodules That Don’t Suck

Git X-Modules lets you enjoy the speed, scale and simple sharing of Git submodules without the bit that drives you to delirious madness

Look out: Git submodules are a trap

It starts out so innocently.

You have two applications. They share a lot of code. The last thing you want is to waste time developing different versions of that code. You want a way to work on everything together.

Well, that’s just what Git submodules are for, right? The shared code has its own repository, which sits inside a parent repository for every application that uses it.

You don’t even have to install anything. It’s already in the tool. You type one command and it’s up and running. Easy peasy.

That might have been as much as you thought about it. After all, this wasn’t the bit that really excited you. Your mind was on what you wanted to build.

It slowly dawns on you that Git is really complicated

And now you – and your whole team – have no choice but to be really good at it.

Much of the difficulty comes down to this: Git submodules don’t just sit inside parent repositories. Every version of the parent repository is tied to a particular version of the submodule.

So, if a developer updates a submodule used in five other repositories, the developer has to update every single one of them.

If that sounds tedious, that’s because it is. But it has to be managed meticulously or projects can break and hours of work can just vanish.

Other times, different sets of changes from different developers wind up in different versions of the submodule, across different instances of the repository. Putting this all together for code review becomes a massive headache.

You might mutter to yourself that the entire point of having version control was to avoid this exact situation. But here you are, juggling multiple versions of the same files.

Remembering the good old days when we all knew what to type

Suddenly, all the commands you used to know must be done differently and with more of them. Here’s just a taste:

The familiar way

Clone

git clone git://example.org/repos.git

Commit

git commit

Push

git push

Create branch

git checkout -b branch

The submodule way

Clone

git clone git://example.org/repos.git
git submodule init
git submodule update

or

git clone --recurse -submodules

Commit

git commit /*in each submodule*/
git add /*in the parent repository for each module*/ 
git commit /*in the parent repository*/

Push

git push /*in each submodule*/
git push /*in the parent repository*/

Create branch

/*Create a branch in each submodule and in the parent repository*/

This all has to be done precisely, and in the precise order too, or else the problems multiply.

It’s so mundane and repetitive that it feels obvious to just write some quick scripts for and be done with it. Except you’ll never really be done with it. You just have a growing list of scripts that need regular attention as projects grow and change.

There’s no getting around the fact that submodules are a very involved way of thinking about projects, which your whole team has to engage with all the time.

And if you’d rather use a graphical tool like Github Desktop? You might be out of luck. Some tools don’t support submodules well. Others don’t support them at all.

Using Git submodules, it doesn’t take a big external shock to derail projects

All you really need is for someone to be distracted or have a bad day.

It’s an expensive way to be miserable

The simple way to start tracking your losses is to add up all the time spent dealing with version control and fixing mistakes. And remember: everyone is losing time to this.

Then ask yourself: what is this doing to your team’s focus and momentum? Their morale and pride in their work?

That’s harder to measure, but it’s still very real.

The pity is that a modular approach should be amazing

There’s so much to like about the idea of separating different projects into their own repository.

It’s fast

Commands take milliseconds, not minutes.
Developers can search and edit a whole repository on their local machine.

It scales

It grows with you from 3 developers to 3,000 and beyond.
There’s no worry about what happens when 100 developers commit to the same repository at the same time.

It’s so easy to share

It makes it simple to work with external contractors without granting access to all your code.
If you want to publish open-source projects without publishing all of them like Richard Stallman, that’s just as simple.

It’s in one place

You always have a single, authoritative version of the code.

These are solid benefits. It’s tough to give them up. So, little wonder that Git submodules still have advocates who will tell you to just deal with it and get used to the headaches and mounting technical debt.

But let’s get real about one thing

Look, it’s not impossible to find developers who get excited by great version control. I mean, we’re some of them.

But most developers aren’t. They’ve worked too hard to build their own areas of focus.

They’ll deal with version control just enough to hold a job down and avoid disaster. But it’s never going to be the thing they’re passionate about, that inspires them to their very best.

It can be thankless too. About the best they can hope for is to do everything perfectly and then see nobody notice because nothing went wrong.

And really.. are developers even wrong to want this off their plate?

Why shouldn’t they have all their attention on their code?

Introducing Git X-Modules

An X-Module sits as an ordinary directory inside a regular Git repository. Developers can treat it just like any other directory, without having to think about it any deeper.

Server-side software keeps this X-Module directory in sync with an external repository, pushing changes in both directions automatically.

What if there’s a conflict? X-Modules accepts one set of changes, turning the other into a pull request.

And if you don’t want to synchronise certain files? You can exclude them either individually, or by extension. So it’s easy not to have those 7-gigabyte .iso files clogging up the bandwidth.

Code review made simple

With X-Modules, all the changes are submitted to one place.
Once they’re approved, they’re synchronised across all repositories automatically.

It’s the modular Monorepo

You’re welcome to think of X-Modules as a kind of monorepo. Because there really is one big repository that everything synchronises to.
You’re also welcome to treat them as separate repositories. You still enjoy all the speed, scale and simple sharing of a modular approach.

Your team already knows how to use it

The best thing about X-Modules is that your team doesn’t have to even pay it much notice.

They just interact with it like any other repository, working exactly the same as all the repositories they’ve used before.

All the familiar commands just work. They can use any third-party tools with no special requirements.

So who else wants the server-side solution that syncs your code and stays out of your way?

Git X-Modules is available as a cloud app for GitHub. You can use it for free while it’s still in beta. Support for GitLab and Bitbucket Cloud will be added soon!
There's also an on-premises app for Bitbucket Data Center.

Seriously, you should try it out.

Text by James Mawson