AWS SnapStart - Part 17 Impact of the snapshot tiered cache on the cold starts with Java 21

Introduction

In the course of this blog series, we measured cold starts with very different scenarios mainly with SnapStart enabled. Now let's explore one more SnapStart detail which is called "tiered caching" of the microVM snapshot. This procedure was briefly mentioned in the article where SnapStart was announced Here is the sentence : With SnapStart, when a customer publishes a function version, the Lambda service initializes the function’s code. It takes an encrypted snapshot of the initialized execution environment, and persists the snapshot in a tiered cache for low latency access. So what this tiered cache might be?

Tiered cache of the snapshot

In our experiment we'll re-use the application introduced in part 9. Let's take GetProductByIdWithPureJava21Lambda function with SnapStart-enabled (but without priming) and 1024 MB memory and measure the cold start for exactly 1 invocation. Let's assume it's a very first execution after the newer version of the Lambda function has been published. My result was 2270.28 ms. The cold start is still big enough. Without SnapStart enabled the result might be in the range of 3100 and 3600 ms.

But what will happen with the subsequent cold start times with SnapStart enabled? Let's see how percentiles change for increasing number of the cold starts.

So, what we observe is that the cold start times reduce the more cold start we experience. After 50 cold starts the effect becomes less and less visible for p50 and after 100 cold starts for p90. This is the effect of the tiered cache for the mivcorVM snapshot in action. The effect of the tiered cache dependings on percentile and is significant (up to 600ms).

If you are interested in the deep details about how Lambda SnapStart (which is currently only available for Java runtime) is implemented and particularly microVM (the whole execution environment) snapshot and its tiered caching works under the hood, I recommend you the talk AWS Lambda Under the Hood by Mike Danilov. There is also a detailed summary of his talk here and additional resources here.

Of course the question was what will happen if we don't invoke my Lambda function for a while and than execute it later? Will the cold start increase. Let's check.

Let's then stop Lambda function execution and then invoke it 30 min later at 21.22 - cold start was 1674.06, then stop again and invoke it at 21:52 - cold start was 1702.17 then the same at 23:00 cold start was 1735.06. So, it got slightly bigger, but we don't observe the worst values from the first executions. Then I stopped Lambda execution for 8 hours and executed it next morning then 15.000 times running into 16 cold start with p50 being 1669.07 and from p90 on 2019.88. So tiered caching effect was still there after so many hours and the p90 and higher numbers didn't look that big as during the first invocations.

To complete the test of snapshot tiered caching I also did the same experiments on GetProductByIdWithPureJava21LambdaAndPriming which uses SnapStart and DynamoDB request invocation priming on top. Let's summarize them in the table below.

So, what we observe is the same as without priming. The cold start time reduces with more cold starts we experience. After 50 cold starts the effect becomes less and less visible and after 80 cold starts the effect becomes negligible for p50 and after 90 cold starts for p90. The effect of the tiered cache dependings on percentile and is significant (up to 500ms).

Of course I had the same question was what will happen if I don't invoke my Lambda function for a while and than execute it later? Let's check.

Let's stop the Lambda function execution and then invoke it 30 min later at 21.22 - cold start was 746.63, then stop again and invoke it at 21:52 - cold start was 617.7 then the same at 23:00 cold start was 673.5. Then I stopped Lambda execution for 8 hours and executed it next morning then 15.000 times running into 17 cold start times with p50 being 723.99 and p90 being 894.05. So tiered caching effect was still there as well after so many hours and the p90 and higher numbers didn't look that big as in the first invocations.

Conclusion

In this article we saw microVM snapshot tiered cache in action for SnapStart-enabled Lambda function (with and without priming) with Java 21 runtime. The conclusion is quite obvious: don't stop by enabling SnapStart and measuring only one cold start time or a couple of them. Yes, the first cold starts take longer, but it gets better with the number of invocations for the same Lambda function version and seems to stay on a good level independent whether you invoked your Lambda for a while or not. I assume exactly the same or very similar effect to be with Java 17 as it's not about the Java version itself but about technical implementation of the microVM snapshot tiered cache done by AWS.