Bespoke Tooling: Building your own hammer

There are lots of awesome tools out there that will help you automate mundane tasks. But there are plenty of reasons why you would want to use custom tooling, too.

Why Custom Toolkits Are Necessary

The reasons I've seen for custom tooling are plenty:

• It may be just to wrap some other tools and create your own zero-config toolkit from that.
• I've been in situations where a specific business use case needed to be validated with high certainty, but the test data was unreliable, so we built our own mocking server.
• I've been in the situation where the same 3 steps for testing my code changes were necessary over and over again – automating those 3 steps allowed for a development experience with quicker feedback loops and ultimately made me faster.

Automation Pays Off More Quickly Than You Think

There is an infamous XKCD post that shows how quickly automating a mundane task pays off

The tricky thing: You barely notice the things that take between 1 and 15 seconds, even if you do them 50 or 100 times per day.

For me personally the big aha-moment was when I started using prettier. It showed me how frequently I spent energy on formatting code. Until then I never considered it a burden to format my code – I love well formatted code so I found that an activity worth spending my time on and always thought I would be doing that just naturally while writing code. Once I was set up to use prettier, I started writing much code in a single line and had prettier format it all for me on save. Suddenly all my mental energy was focussed on the code itself and not its formatting.

Copy & Paste Mixed With Search & Replace is not Tooling

For a while when we created a new micro-service, we would go about it the following way:

1. copy & paste an existing deployment file
2. search & replace the name of the micro-service
3. adjust any additional configuration that's service specific (e.g. database connections)
4. copy & paste this file for the other environments
5. make environment specific adjustments
6. wait for a code-review for someone to double-check your deployment file
7. apply the deployment file in the test environment
8. test if the micro-service works as expected
9. repeat 7 and 8 for the other environments

For us a big milestone was the auto-generation of these deployment files. Suddenly it was impossible to make copy & paste mistakes. Creating "another one of those", in our case a worker deployment, was possible with zero config. A potential source of error was completely eliminated! The important thing about zero config isn't that you're avoiding a config file. It's that you're avoiding knowing about that file and making mistakes with it.

Search & Replace is easy when you need to do something once. But every time you need to repeat this process, you need to make sure you didn't miss anything and need a complete review.

Automating this process with real tooling suddenly made it easy to create a new service. Something nobody would ever need to think twice about.

Good Automation Tooling Compounds

Once the file generation was automated, it became easy to build on top of that for other things, too. Suddenly it became simple to automatically deploy the service. We gained the confidence to automatically deploy it to all environments very quickly.

Later on we would know how to automatically build different deployment files, depending on exactly what kind of service we were talking about. We were able to build on top of that to automatically update configurations for all services. And we were able to build on top of that to improve how we were handling secrets – something we had previously spent many hours debugging.

When you slowly build your toolkit, you can make sure that every helper you write does only one thing and it does it well. That allows you to compose those tools. And more importantly, it makes things easy to do that were previously hard, like creating a new micro-service.

Making the Right Thing The Easy Thing

I love taking shortcuts. If I can leave out some detail, I will. I know there are many like me. But I also know that the shortcut often has a price: You might encounter technical debt. Most of us have seen that situation when tests were skipped, but there are countless other examples when the developers new better and still took the shortcut.

Tooling can make the right thing the easy thing. The way I made sure I would write tests? I made it easier to write a test than to test my service manually! Any call would have 2 phases: input to data system and data system to output. Once I realised that, I made certain I would easily generate a JSON file for input (same file for manual and automation test!), mock the data source (this part was actual work) and use snapshot testing to check the output. 10 seconds to write a simple test case. 10 more for every corner case.

When you make the right thing the easy thing, people will go down that route.

The 3 Rules Of Bespoke Toolkits

1. Pick a routine task
2. Automate one and only one thing about it
3. Do it in a way that encourages doing The Right Thing

Then Repeat.

Great packages like yargs, execa, fs-extra and many others will help you build that toolkit.