It's been so long since I have written a technical post, that I feel like I have completely forgotten how to write one. Deep breath and let's start.
Historically in most places I worked for, there was always a need for a scheduling service. These could come into many different flavours:
- Embedded into an app
- Implemented through a queuing mechanism
- Triggering jobs or services
- Notifying through a webhook
Some of them can be solved with traditional ways i.e cron jobs, others need a more complex solutions.
But when this discussion starts, it is always a heated one with lots of opinions on the how(s) and why(s).
I always found it fascinating, when a certain space has so many opinions. In these cases, I feel the urge to add my opinion to the pool.
This is just a mental exercise, I like to overthink a problem and try to create a solution that makes sense to me.
So I came up with this
Initially, I tried to make the components to run on different threads. Which worked, but something bugged me.
Then I thought, it would be nice, if individual component could scale based on its needs, without the need to redeploy all of them.
So I split everything, on it's own executable. They didn't have to communicate with each other anyway. Every binary has a single specific responsibility. At this stage I am using the database to sort any data race conditions. But the actual datastore shouldn't matter that much, you could use one, or many.
It's definitely a project I want to see through and create something stable, but as typically things that are not used or are intended to be used in production, rarely survive as actual projects. But, they do make excellent references as training material.
This is the repository if anyone is interested. link here