First impressions of the fastest JavaScript runtime for Lambda

#aws #javascript #lambda #serverless

I thought Lambda needed a specialised runtime. One that works well with its resource-constraint execution environment. I even floated a few ideas in the past but sadly I don’t have the chops to make them happen myself.

So I was pleasantly surprised when AWS open-sourced the LLRT runtime for JavaScript [1]!

What is LLRT?

LLRT, or Low Latency Runtime, is a new and experimental JavaScript runtime for Lambda. It promises 10x faster startup time. Which should significantly help with the dreaded Lambda cold starts.

Naturally, I had to test it out for myself and see if the hype was real.

My first impressions of LLRT

In my limited tests, the init duration for a simple function went from ~750ms to 55ms. Which was very impressive! The function under test only uses the DynamoDB client from the AWS SDK v3 and makes one request to DynamoDB.

Along the way, I also discovered several limitations:

It only supports ESM modules.
It doesn’t work with the popular middy [2] middelware engine. Because LLRT hasn’t implemented the node:stream API. See the API compatibility [3] page for the list of supported APIs.
It doesn’t work with the Lambda Powertools [4]. Because the LLRT doesn’t allow importing the node:console method in userland. See Andrea Amorosi’s response here for more details.

LLRT is not ready for prime time yet. And given that it’s been in the works for almost two years, we should set expectations accordingly.

It’s probably not going to be production-ready in the coming months. But the potential is there and I’m really excited about it!

For one thing, it might finally stop people talking about cold starts. As I said on LinkedIn [5] yesterday, most people are overthinking about Lambda cold starts. If you’re not sure if Lambda cold starts is likely a problem for you, then go read the LinkedIn post and come back here.

Back to LLRT.

I later spoke with Richard Davison, the creator of the LLRT project.

I wanted to learn more about the project and the design decisions behind it.

What made it start so much faster?

What trade-offs did they make?

LLRT is the answer to the question “What would a purposely built JavaScript runtime look like for Lambda.”

A lot of people see that “It’s built in Rust” and automatically assume that’s why it’s fast. But it’s more than that.

When it comes to performance optimizations, it always boils down to what you let go. If you choose to do everything the incumbent does, then you won’t make any significant performance gains.

No JIT compilation

With LLRT, they chose to not include a JIT compiler. Because it’s focused on Lambda’s resource constraint and short-lived execution environments.

As a result, LLRT is likely less performant than the Node.js runtime for CPU-intensive tasks.

However, most Lambda functions do not perform CPU-intensive tasks. Instead, they tend to be IO-intensive.

And the Lambda execution environments are short-lived. So a JIT compiler would have been less effective at optimizing hot code paths.

At the same time, a JIT compiler comes with significant startup costs. It also introduces occasional latency spikes when it needs to evict cached items.

So it appears a sensible trade-off for LLRT to not include a JIT compiler.

QuickJs + Rust

The QuickJs engine [6] plays a crucial role in LLRT and its outstanding performance. Another reason why LLRT is fast is because they wrote all the APIs in Rust.

As much as possible, the team wants to stay in native code to guarantee a strong performance. Bun [7] took the same approach and implemented all the APIs in a system language called Zig [8].

The downside to this approach is that it’s harder for contributors to get in on the action. There are a lot fewer Rust developers than Node.js developers. And even fewer Rust developers who are interested in a JavaScript runtime.

LLRT vs Bun (and other JS runtimes)

LLRT is different from other JavaScript runtimes. It’s not a general-purpose runtime for JavaScript. It doesn’t have to worry about running in the browser or on mobile phones.

Instead, it’s solely focused on the Lambda execution environment.

This allows them to make decisions that just wouldn’t make sense with Bun or Node.js. Decisions such as not including a JIT compiler. Or which of the JavaScript APIs do they implement first, or at all?

The goal is to eventually become WinterCG compliant. But we don’t have to wait for that to start using LLRT. For LLRT to be useful (but not perfect!), it just needs to support the AWS SDK and a few popular libraries.

Summary

To summarize, LLRT is an exciting new runtime for JavaScript.

It’s purposely built for Lambda.

It’s not intended as a general-purpose runtime for JavaScript.

It makes design trade-offs (such as not having a JIT compiler) to achieve an optimal cold start time.

It uses the QuickJs engine and implements all the JavaScript APIs in Rust.

To learn more about LLRT and how to contribute to it, please check out my conversation with Richard on YouTube [9].

I will be covering LLRT in my upcoming workshop, including how to use it with the Serverless Framework and CDK. If you wanna take your serverless game to the next level, then you should check it out! More information is available on the course page [10].