GraphQL Performance Benchmarks - Hasura vs DIY Node.js with Dataloader

Hasura enables you to instantly create a production-grade GraphQL API with authorization and other essential features, such as event triggers, real-time capabilities, and much more for your data without writing code.

Once you connect it to your database, Hasura automatically generates an API with the following:

• a schema
• queries
• mutations
• subscriptions

As a result, you save significant time and effort since you don’t need to write your logic for basic CRUD operations manually.

The first question people have about the automagic is - how does Hasura perform? This article aims to answer the question by benchmarking Hasura against a handwritten Node.js GraphQL server that uses DataLoader.

The comparison proves that Hasura performs better (in some cases significantly) while drastically reducing the effort and resources required to build an API.

Benchmark setup

We took these benchmarks on Google Cloud Run configured with 4 CPU cores and 16 GB of RAM. For the database we used the Chinook dataset on Neon serverless Postgres 15. We used GraphQL Bench and k6 with Hasura 2.18.0 and Node.js 18.

Three tests were ran:

1. For every artist, get their albums and the tracks in those albums with two thousand requests per second (RPS) for ten seconds.

2. Get an album by primary key with ten thousand requests.

3. Get an album again by primary key, except with 100 RPS for 5 seconds, then 1000 RPS for 5 seconds.

Results

Fetch All Artists With Their Albums and Tracks

Average Hasura requests per second (RPS) was 29.17 while Node.js was 8.05 RPS.

Hasura

Node.js

Fetch Album by Primary Key Fixed Requests

Average Hasura latency was 64.85 while Node.js was 68.65.

Hasura

Node.js

Fetch Album by Primary Key Fixed Multistage

Average Hasura latency was 60.68 while Node.js was 63.21.

Hasura

Node.js

Takeaways

N + 1 Problem

When writing GraphQL APIs, one of the most significant performance issues is N + 1 queries. For our artist benchmark, each artist has to fetch the related albums. If implemented in a naive way, we would have to make N number of SQL requests, one for each artist. 100 artists would result in 100 requests for album data! To solve this in Node.js, we use the Dataloader library. Dataloader allows us to batch requests, so instead of 100 requests, one request can get all the associated albums. The tradeoff is that you must manually add these dataloaders everywhere that needs them, which can sometimes be unclear. This is on top of the work you already have to do creating your server.

Dataloader is primarily there to solve batching, but if you want it also to do caching, the complexity increases further. Let’s say a query fetches a track by trackID and later fetches the same track by albumID. By default, these are two separate Dataloader caches resulting in two seperate requests for the same data. To fix this, you must manually prime the trackID and albumID cache, depending on which one you just fetched. So as different ways to fetch the data grows, so does your code if you want to stay performant.

Please visit the source code to see how this code can exponentially grow.

Here is a sample of the Album resolver:

Album: async (_parent, _args, context: Context, _info) => {
    const albums = await context.getAllAlbums.load("1");
    for (const album of albums) {
        context.getAlbumsById.prime(album.AlbumId.toString(), album);
    }
    const albumsByArtistId = keyByArray(albums, "ArtistId");
    for (const [ArtistId, albums] of Object.entries(albumsByArtistId)) {
        context.getAlbumsByArtistId.prime(ArtistId, albums);
    }
    return albums;
},

Hasura acts as a SQL compiler. Because it knows how your data is structured, queries can be compiled into a single SQL statement instead of multiple. See this blog post for more information on how Hasura solves the N+1 problem. We can see the benefit of this in the "Fetch All Artists With Their Albums and Tracks" benchmark, Hasura was 3.6x faster!

Read more:

• Architecture of a high performance GraphQL to SQL engine
• Blazing fast GraphQL execution with query caching and Postgres prepared statements

Automagic generation

The Node.js sample GraphQL server demonstrates the amount of code required just for the basic functionalities such as queries. Adding more features such as mutations, subscriptions, or authorization requires even more code and time.

In your own applications you may spend up to 50 - 80% of development on CRUD. Hasura saves you a significant amount of time from writing the basics of a production server and comes with features such as authorization, security, event triggers, and API joins built-it.

Read more:

• How Hasura Works Docs

• Hasura Design Philosophy

Conclusion

Benchmarks are challenging, and these results are not scientific. We showed that in a previous blog post things like CPU power management could play a significant role.

As part of this benchmarking series we discovered possible places we can speed up Hasura's HTTP stack. So overtime, your APIs will get even faster with no user change needed.

We hope we demonstrated that automatically generated Hasura GraphQL API performed very similarly and better to a handmade GraphQL server with no development time and with added features like authorization on top for free!