There are numerous blog posts about using Bazel or Bazel vs something, etc. Unfortunately I have not managed to find any good articles from people using Bazel for a side hustle project, especially something that’s maintained by just one or two people. I’ve recently published a job aggregator website for the UK, called NoxidJobs and I’ll use my findings from building it to try and give you all the information needed to make your decision on whether Bazel is the right tool for your side hustle project or not.
Before we start discussing why Bazel might or might not be useful for your side hustle project, let’s briefly cover what Bazel is. Bazel is an open-source (open-sourced in 2015) version of the internal build system used at Google, called Blaze. Both of these systems are artifact-based and their build files are written in a declarative manner, using the Starlark language. If you’re unfamiliar with these concepts, you can learn more about them here.
Some of the key features of Bazel are:
- Dependency tracking. It uses a directed acyclic graph to track dependencies and only rebuilds what is necessary, which significantly speeds up the build process, especially for large projects. The dependency tracking can be especially useful if you employ multiple languages in your project and have dependencies between them.
- Reproducibility. Bazel aims to provide consistent and reproducible builds across different environments and platforms. This ensures that builds are not affected by the local development environment and minimizes unexpected issues when moving code between different systems.
- Caching. Bazel employs a robust caching mechanism. This means that if a build has been done once, the results can be cached and reused on subsequent builds, reducing the time required for incremental builds.
- Extensible. Bazel is highly extensible and allows you to create custom build rules for your specific project needs. This flexibility makes it adaptable to a wide range of use cases.
- Multi-platform and Multi-architecture support. Bazel can handle cross-compilation and multi-platform builds, making it suitable for projects that target a variety of platforms and architectures.
- Remote Execution. Bazel can utilize remote build execution, which means that builds can be offloaded to remote servers or cloud infrastructure for faster build times and resource management.
Yes, some of these highlights scream large-scale projects :) so let me cover why I’ve considered using Bazel for my side hustle project without further ado. First, I’ve used similar systems in fairly large organisations and I really enjoyed it. There are quite a few out there and are well covered, so we’ll just consider Bazel here. Second, I’ve initially considered using 3 languages for my project - Go, Python and JavaScript (I’ve ended up dropping Python at the end). Third and probably most important, I’ve decided to build a distributed system and deploy it on Kubernetes. Although in the initial release, the end-user interface is extremely simple, there is quite a bit going on under the hood and I wanted to be able to extend the system quickly and easily, adding more features such as authentication, the ability to post jobs, custom statistics pages, etc and at the same time I wanted to be able to do things like A/B testing, etc. So if you’re considering building a monolithic application, using something like Django and running it on a single VM or something similar, let me say that Bazel is definitely an overkill for this. If you’re considering building something a little bit more sophisticated, then let’s continue with the rest of the article :)
Let’s get the cat out of the bag before we continue and let me say that I’ve used Bazel up until just before pushing out the first version of NoxidJobs. Before publishing the first version, I’ve made the switch to a much simpler tool called Task. But before we continue I must admit that I’ve not completely given up on Bazel and I might end up revisiting using it in the future, so the story doesn’t end here and furthermore the point of the article is not to discourage you to use Bazel, but to give you enough information on whether it’s right for you from the perspective of using it for a single-person (or two to three people) side hustle project.
First off, let’s get the Remote Execution out of the way. Since this is a single person side hustle, I have not even considered it. Even with more people that is not something that will be of concern as the code base cannot possibly get large enough to make it useful.
Now to the things that I found useful (or thought were useful :sweat-smile:):
- Reproducibility was probably the feature that I found most useful as I did not have to worry about differences between building the source code and container images on my Mac versus GitHub Actions, which is what I ended up using for the CI/CD. This actually saved me some time as I didn’t have to worry about testing my Actions much. I was fairly confident that what I build on my MacBook will work in the Action with fairly good confidence.
- Dependency tracking was quite useful as I managed to create some shared libraries to use for the content scrapers and save quite a bit of time. Having said that, when switching to the new build tool, I managed to achieve the same using Go workspaces.
- Caching was great as well. That’s until I started building the CI/CD, but we’ll get to this later.
- Multi-platform and Multi-architecture support was not something I thought about initially, but I ended up switching to an Apple silicon for my personal computer and Bazel made it fairly straightforward. The only thing was that I had to disable CGO, otherwise things got extremely complicated quite fast with having to create a custom toolchain, etc. Having said that I ended up doing the same when building with Task due to the same issues (linking for Alpine :sweat-smile:), so I don’t consider that being an issue.
In reality, the reproducibility was the best feature for my purposes. Having the exact same tools, pinned at the exact same versions was quite useful and saved me fair bit of time debugging issues when building the CI/CD pipeline. The dependency tracking was quite useful especially for tracking dependencies between the JavaScript code as I’ve been experimenting with Next.js and other frameworks, etc (not yet published live at the time of writing this article) and Go. It did save some time rebuilding things, but after I made the switch to Task, I managed to shave some time off in other places so this was not crucial. Caching is always useful, but Bazel has a very aggressive caching and the caches grow quite quickly. For a fairly small project like NoxidJobs (at the time of writing), the generated cache was enormous. I’ve quickly reached the 10G limit I had for my GitHub account and the time it took to restore the cache for each job quickly reached around 3 minutes, bringing the total time to run the tests for each PR to ~5-6 minutes. I tried moving the caches to a GCS bucket, but that made it even worse in terms of time and being a single person side hustle without a budget, etc I was out of other options. After the switch, I managed to split linting, testing and coverage upload (to codecov.io) for JavaScript and Go into separate jobs that ran in parallel and each had their own caches. With this approach, I managed to bring down the total time to run the tests to less than a minute if there were no changes to the external dependencies (downloading dependencies takes a bit of time). While Bazel’s multi-platform and multi-architecture support is essential for C and C++ projects that need to be developed and maintained for diverse environments (most probably Google’s use case:), I’ve used Go for most of the backend and services which is fairly easy to cross-compile, has easier dependency management, etc that made it as easy to use the standard Go tooling. As for the JavaScript, I’ve struggled quite a bit to integrate the tooling with Bazel and didn’t manage to get everything working the way I wanted to until I made the switch away from Bazel. This is mostly due to the fact that I don’t have much experience with the JavaScript (and React and Next.js, etc) tooling which brings me to the next point - you need to understand the underlying tooling quite well if you decide to use Bazel. That’s one of the main things that made me do the switch at the end. I have fair bit of experience with Backend development and Platform Engineering/SRE, but with frontend, not so much. With the tooling around the languages and technologies that I was familiar with, I had no issues to fiddle with and put in some time to write custom build rules and fix things, but I have not used any of the frontend tooling for years before I started working on NoxidJobs and I’ve struggled quite a bit getting anything up and running. Another thing that made me switch away from Bazel was the fact that most of the rules for things such as container images are not included in the build system and are not maintained by Google. In fact when I started building the project, I started off with using the rules_k8s that looked kind of official, along with the rules_docker. Both of these projects were archived, with the rules_k8s being archived without any warning, explanation or a real alternative. The only alternative I could find was the rules_gitops which was published and maintained by Adobe, but at the time of writing they were still using the deprecated rules_docker which were archived shortly after the rules_k8s. That left me with the option to continue using the deprecated rules or write my own from scratch. Writing your own is probably the best option and is what most large organisations do, but since that’s for a single-person side hustle and speed and ease of development is a major factor, I’ve decided to make the switch away from Bazel. If you have significant experience with using Bazel and all of the underlaying tools required to build your project this probably won’t be a factor, so you might still want to consider using it.
Using systems and tools that were ready made and easy to integrate was of great importance for building my project as I don’t have much time to spend on it, so using tools such as Dependabot and codecov was quite important. Unfortunately this proved difficult because Bazel has its own opinion about managing external dependencies and even though Gazelle helps a lot to integrate the Go modules with Bazel and can update the dependencies it was still hard to integrate with everything. For example when I initially set up Dependabot, it only updated the go.mod and go.sum and ran the tests but because it didn’t update the Bazel deps and it’s not very flexible I couldn’t run Gazelle to update them and had to go through each of the PRs, checkout the branch, run Gazelle, push the changes and only then I’d find out if the tests pass or not. With the standard tooling, Dependabot updates the dependencies, opens the PR and all tests run automatically showing any failures without the need for manual intervention. Of course there are tools that already support Bazel such as Renovate, so it depends on what tools you choose and which tools are familiar with. In my case it seemed like another overhead in terms of time (which was already quite limited to start with) to integrate another tool that I’m not familiar with just so I can keep using Bazel.
I’ve managed to replicate some of the things I really liked when using Bazel with other tools, such as using the Go workspaces for building common libraries and dependency management (that meant that I could no longer build the Go applications inside of the Docker builder containers, but this is probably a story for a separate article) and having dependencies between the JavaScript code and the Go code using the task dependencies or just calling subtasks where required. So I’m still using a monorepo with dependencies between the different applications, libraries, etc and albeit being a lot simpler it enabled me to bootstrap my side hustle project more quickly than with Bazel and helped me speed up the tests. Having said this, I’m still considering revisiting Bazel in the future as I think it is an awesome tool but that will only happen when and if my codebase grows in size and complexity and when I gain enough knowledge around all the tooling that I’ll need to use to build the whole project, so I’m comfortable creating new rules, etc for things that are specific for my needs. If you’re confident enough with all the tooling and don’t mind spending the extra time to fiddle with the build system, then probably Bazel will be the right choice for you. Let me know your thoughts in the comments and it would be great to hear your experience of using Bazel for small, single-person projects.
Top comments (0)