Developer Q&A: Monitoring global API latency with chronark

Tinybird: Hey chronark! Before we jump in, can you tell us a little bit about yourself?

chronark: Of course! First of all, my real name is Andreas, but I pretty much go by chronark everywhere (including Twitter: @chronark_ and Github: chronark). 

I live in Nürnberg (Nuremberg to the English speakers among us), and for my day job, I’m a software engineer at Upstash. During my off hours, I build things. Most of what I’ve built and am building is here: chronark.com

Most recently I’ve been working on a project called Planetfall.

Tinybird: What’s Planetfall?

chronark: Planetfall is a way to understand the true performance of your API by monitoring its global latency. So it can tell you how long your APIs take to respond all over the world, from Frankfurt to Washington DC to Singapore. 

I haven’t technically incorporated it yet, but that’s in the works. I had the idea for it when I was on a vacation in Turkey in September of last year, so I moved pretty quickly to build it. 

You can check it out here: planetfall.io

Tinybird: Why did you decide to build it? Said differently, why does Planetfall matter to the typical dev?

chronark: We’re a global society with global consumers. It’s no longer good enough for your application to only serve a single region quickly while ignoring the rest. If you only focus on performance in a region or two, you’re massively limiting your reach and potential customer base. 

We’re a global society with global consumers. It’s no longer good enough for your application to only serve a single region quickly while ignoring the rest.

Plus, you’re potentially opening yourself up to bad user experiences from global users. Moving forward, app builders need to be thinking about global low latency.

The mission that Planetfall has is to make it very easy for developers to see where their APIs underperform so they can debug those latency issues and ensure a great user experience all over the world.

Tinybird: Are you building Planetfall yourself?

chronark: Yep, just me! Though I must say that Dom Eccleston, Dom Sipowicz, and Steven Tey have all been amazing beta testers with really valuable feedback, so h/t to them!

Tinybird: What problems did you face as you were building Planetfall that ultimately led you to Tinybird?

chronark: So the challenge with Planetfall is that the more active users it gains, the more checks per second that it needs to run and store. Running the checks is pretty easy, since they’re just running in lambda functions, and that scales, so I’m not worried much about that. 

But storage is a different story. I was using MySQL to store all those checks. And what I would do is filter and download all those checks and perform the aggregations on the client side. At a certain point, MySQL just can’t scale for that use case. It would be kind of ironic to build an API latency monitoring service and have slow APIs 🤣.

I was using MySQL to store all my API checks. At a certain point, MySQL just can't scale for that use case.

So I looked at a few different time series databases to try to help me with this, but none of them were really a great fit.

So naturally, I asked Steven Tey, and he recommended Tinybird.

Tinybird:  We should be paying Steven Tey.

chronark: Probably.

Tinybird: So now that you’re using Tinybird, how does your solution look?

chronark: So I’m actually still using MySQL (via Planetscale) but in a different way. I basically have a hybrid approach where I store my check results in MySQL and also send the data needed for aggregations to Tinybird.

I was pushing everything to Tinybird, but since I also store the response bodies, it was getting too expensive. That’s probably because I wasn’t filtering correctly, so in the future, I want to go back to storing checks only in Tinybird so I can remove that MySQL dependency, but for now it works!

Tinybird pretty much powers everything except that raw storage bit, from the public status pages to usage-based billing. And of course, I use Tinybird to calculate the daily and hourly latency percentile metrics and publish those as APIs.

Tinybird powers pretty much everything except raw storage, from public status pages to usage-based billing and the latency percentile metrics that power Planetfall.

Here’s the way it all shapes up in a diagram:

Tinybird: Anything you particularly love about the way you’ve built this that you want to share?

chronark: Yes. I think the custom, fully-typed Tinybird client I built is pretty cool. Here’s an example of how that gets used.

I’m a bit of an SQL noob, so there’s nothing really amazing about my code, but I do love the Pipe I wrote for billing. Seriously, usage-based billing has never been this easy:

SELECT
   teamId,
   countState() AS usage,
   toYear(time) AS year,
   toMonth(time) AS month,
   toDayOfMonth(time) AS day
FROM checks__v1
GROUP BY
   teamId,
   year,
   month,
   day

Here's how the data flow looks in Tinybird for that Pipe.

It's a great use of Materialized Views. I wrote a Pipe (aggregate_usage__v1) to get yearly, monthly, and daily aggregates of checks grouped by teamId. I materialize the results from this Pipe into a Materialized View (daily_usage__v1) so the aggregation gets stored at ingestion time. Then I have another published Pipe (get_usage__v1) that queries that Materialized View and is published as an API. I have cron job that calls that API once a month, generates the invoice for each team, and then sends it through Stripe. I also periodically call the published API to make sure that teams haven't exceeded their usage limits.

This Pipe basically powers my Stripe integration and my customer-facing billing dashboards like this one: 

Tinybird: What have you loved most about working with Tinybird?

chronark: The developer experience. I've been able to move so much faster by writing Pipes in Tinybird than I could ever have by pulling data manually from MySQL and transforming it myself.

I can iterate crazy fast with Tinybird and validate new ideas almost immediately. For example, I wanted to build some kind of audit log for CRUD operations on resources (such as a team member being added). 

With Tinybird, I can iterate crazy fast and validate new ideas almost immediately.

Initially, I thought I’d build this in MySQL, but then I just sent those events to a new Tinybird Data Source using the Events API. I love that I can design a schema in Tinybird if I want to, but I can also use the Events API’s schema inference in dev and then set up a more precise schema in production. And it’s basically just a one-liner in my code.

I haven’t actually gotten around to building this feature yet, but I know the data is there and I can refine it further when I need it.

Also, the support is 🧑‍🍳🤌😘. I love you guys. You proactively reached out to help me spend less money with Tinybird.

I should also highlight how easy it is to learn. I’ve basically never written SQL before, but the Tinybird UI made it so easy to get started. The Pipes concept - where you can just chain together these little SQL nodes - is so much more approachable than doing a bunch of joins and nested CTEs and stuff like that.

I'm a bit of an SQL noob, but the Tinybird UI made it easy to build usage-based billing from scratch. I've been able to move so much faster writing Pipes than I ever could have by pulling data manually from MySQL and transforming it myself.

Tinybird: How do you get data into Tinybird?

chronark: I exclusively use the Events API. It’s such an easy way to send data. It’s just an HTTP request and, boom, you have data in Tinybird.

Tinybird: What do you feel like Tinybird offers you that you can’t get anywhere else?

chronark: First of all, the ability to just build and not worry about maintaining or scaling anything is huge. But on a more personal level, I know that Tinybird is full of kind, smart developers who care and have my back. You really get the feeling that Tinybird wants you to win and isn’t interested in just taking your money.‍

Tinybird: 🥹‍

Tinybird: So what’s next for Planetfall? What’s on the roadmap?

chronark: Planetfall is starting to get to a place where I’m more comfortable showing it off. The platform is stable, and I’m adding new features.

Currently, Planetfall offers latency checks from AWS Lambdas, Vercel Edge Functions, and all regions from fly.io. It also includes a free hosted status page to show your customers how fast your API is all over the world.

The next step is to really focus on providing the best possible debugging experience. This is mainly a UI effort, and I’m actively collecting feedback from people to understand what they need and what is intuitive.

I also want to provide an API to provision checks automatically or run them specifically in your CI, which would be cool.

Tinybird: How does Tinybird factor into that work?

chronark: I will definitely keep building new features with Tinybird. It’s too easy not to. I don’t have any other projects in mind right now, but I’m sure that Planetfall isn’t the last thing I will build with Tinybird 😉.

I will definitely keep building with Tinybird. It's too easy not to.‍

Tinybird: Who do you think benefits most from using Tinybird?

chronark: Anyone who wants to go fast. It’s so easy to move quickly and experiment without risk or large upfront costs. Tinybird moves away a lot of the obstacles that can prevent you from building new features.‍

Tinybird: What would you tell something who’s weighing Tinybird versus another product? Any advice you’d offer?

chronark: Honestly, I didn’t really try anything else other than MySQL. I was hooked on Tinybird. It had everything I needed to start and move quickly. So yeah, just try it! Build the simplest thing you possibly can, see what it can do, and go from there. You don’t need to go all in to start, and you can build pretty powerful stuff even on their free plan.