AWS SnapStart - Part 15 Measuring cold and warm starts with Java 21 using different synchronous HTTP clients

Introduction

In the previous parts of our series we measured the cold starts of the Lambda function with Java 21 Corretto runtime without SnapStart enabled, with SnapStart enabled and also applied DynamoDB invocation priming optimization with different Lambda memory settings, deployment artifact sizes and Java compilation options.

In this article we'll now add another dimension to our measurements : the choice of HTTP Client implementation. This is also interesting, because starting from AWS SDK for Java version 2.22 AWS added support for their own implementation of the synchronous CRT HTTP Client. The asynchronous CRT HTTP client has been generally available since February 2023. In this article we'll explore synchronous HTTP clients first and leave asynchronous ones for the next article.

Measuring cold and warm starts with Java 21 using synchronous HTTP clients

In our experiment we'll re-use the application introduced in part 9 for this which you can find here. There are basically 2 Lambda functions which both respond to the API Gateway requests and retrieve product by id received from the API Gateway from DynamoDB. One Lambda function GetProductByIdWithPureJava21Lambda can be used with and without SnapStart and the second one GetProductByIdWithPureJava21LambdaAndPriming uses SnapStart and DynamoDB request invocation priming. We give both Lambda functions 1024 MB memory.

As we did our measurements for cold and warm start with Java 21 using different Lambda memory settings and cold and warm start with Java 21 using different compilation options among others, we have always used the default HTTP Client implementation which is Apache HTTP Client (we'll use the measurements for the comparison in this article), now we'll explore 2 other options as well.

There are now 3 synchronous HTTP Clients implementations available in the AWS SDK for Java.

1. Url Connection
2. Apache (Default)
3. AWS CRT

This is the order for the look up and set of synchronous HTTP Client in the classpath.

Let's figure out how to configure the HTTP Client. There are 2 places to do it : pom.xml and DynamoProductDao

Let's consider 3 scenarios:

Scenario 1) Url Connection HTTP Client. Its configuration looks like this:
In the pom.xml the only enabled HTTP Client dependency has to be:

     <dependency>
        <groupId>software.amazon.awssdk</groupId>
        <artifactId>url-connection-client</artifactId>
     </dependency>

In DynamoProductDao the DynamoDBClient should be created like this:

DynamoDbClient.builder()
    .credentialsProvider(DefaultCredentialsProvider.create())
    .region(Region.EU_CENTRAL_1)
     .httpClient(UrlConnectionHttpClient.create())
    .overrideConfiguration(ClientOverrideConfiguration.builder()
      .build())
    .build();

Scenario 2) Apache HTTP Client. Its configuration looks like this:
In the pom.xml the only enabled HTTP Client dependency has to be:

     <dependency>
        <groupId>software.amazon.awssdk</groupId>
        <artifactId>apache-client</artifactId>
     </dependency>

In DynamoProductDao the DynamoDBClient should be created like this:

DynamoDbClient.builder()
    .credentialsProvider(DefaultCredentialsProvider.create())
    .region(Region.EU_CENTRAL_1)
     .httpClient(ApacheHttpClient.create())
    .overrideConfiguration(ClientOverrideConfiguration.builder()
      .build())
    .build();

Scenario 3) AWS CRT synchronous HTTP Client. Its configuration looks like this:
In the pom.xml the only enabled HTTP Client dependency has to be:

     <dependency>
        <groupId>software.amazon.awssdk</groupId>
        <artifactId>aws-crt-client</artifactId>
     </dependency>

In DynamoProductDao the DynamoDBClient should be created like this:

DynamoDbClient.builder()
    .credentialsProvider(DefaultCredentialsProvider.create())
    .region(Region.EU_CENTRAL_1)
     .httpClient(AwsCrtHttpClient.create())
    .overrideConfiguration(ClientOverrideConfiguration.builder()
      .build())
    .build();

For the sake of simplicity, we create all HTTP Clients with their default settings. Of course, there is the optimization potential there to figure out the right HTTP Client settings.

The results of the experiment below were based on reproducing more than 100 cold and approximately 100.000 warm starts with experiment which ran for approximately 1 hour. For it (and experiments from my previous article) I used the load test tool hey, but you can use whatever tool you want, like Serverless-artillery or Postman. I ran all these experiments for all 3 scenarios using 2 different compilation options in template.yaml each:

1. no options (tiered compilation will take place)
2. JAVA_TOOL_OPTIONS: "-XX:+TieredCompilation -XX:TieredStopAtLevel=1" (client compilation without profiling)

We found out in the article Measuring cold and warm starts with Java 21 using different compilation options that with them we've got the lowest cold start times.

Let's look into the results of our measurements.

Cold (c) and warm (m) start time with compilation option "tiered compilation" without SnapStart enabled in ms:

Cold (c) and warm (m) start time with compilation option "-XX:+TieredCompilation -XX:TieredStopAtLevel=1" (client compilation without profiling) without SnapStart enabled in ms:

Cold (c) and warm (m) start time with compilation option "tiered compilation" with SnapStart enabled without Priming in ms:

Cold (c) and warm (m) start time with compilation option "-XX:+TieredCompilation -XX:TieredStopAtLevel=1" (client compilation without profiling) with SnapStart enabled without Priming in ms:

Cold (c) and warm (m) start time with compilation option "tiered compilation" with SnapStart enabled and with DynamoDB invocation Priming in ms:

Cold (c) and warm start (m) time with compilation option "-XX:+TieredCompilation -XX:TieredStopAtLevel=1" (client compilation without profiling) with SnapStart enabled and with DynamoDB invocation Priming in ms:

Let's visualize our measurements for p90 and draw the conclusions. "SLA" is abbreviation for the compilation option "-XX:+TieredCompilation -XX:TieredStopAtLevel=1" and "tiered" stays for the "tiered compilation".

Conclusion

Generally speaking, as we've already figured out in Measuring cold and warm starts with Java 21 using different compilation options, "tiered compilation" is the preferred choice over "-XX:+TieredCompilation -XX:TieredStopAtLevel=1" (client compilation without profiling) in case SnapStart enabled and other way around in case if SnapStart isn't enabled, but the results are really close to each other.

In terms of the HTTP Client choice, AWS CRT HTTP Client is preferred choice in case SnapStart isn't enabled or SnapStart is enabled but no priming is applied. In case of priming of the DynamoDB invocation, the results in terms of the cold starts for all 3 HTTP Clients are close to each other as the initialization of the DynamoDB Client with the HTTP Client and the most expensive first invocation (priming) happens already during the deployment phase of the Lambda function and doesn't impact the further invocations that much. The Apache HTTP Client is probably the most powerful choice but it shows the worst results for SnapStart not being enabled.

The warm execution times are more or less close to each other for all 3 HTTP clients and compilation options.

Can we reduce the cold start a bit further? From our article Measuring cold starts with Java 21 using different deployment artifact sizes we know that smaller deployment artifact sizes lead to the lower cold start times. The usage of AWS CRT HTTP Client adds 18 MB to the deployment artifact size for our sample application (total size 32MB versus 14 MB for URL Connection and Apache HTTP Clients). If we look into the deployment artifact with AWS CRT HTTP Client, we'll discover the following additional packages for each operating system : linux, osx and windows.

If we take a look into those folders, we'll see for example the following content for the linux folder (the same applies for windows and osx folders) :

As we see the content of such folders is natives file for each operating system and processor architecture: for osx it's libaws-crt-jni.dylib file, for windows - aws-crt-jni.dll and for linux - libaws-crt-jni.so. If we already know that we'll run our Lambda only on Linux x86 architecture, we can delete the osx and windows folders completely and subfolders for arm architecture in the linux folder. This will reduce the deployment artifact size from 32 to 19 MB for AWS CRT HTTP Client and further reduce the cold start time a bit.

The choice of HTTP Client is not only about minimizing cold and warm starts. The decision is much more complex end also depends on the functionality of the HTTP Client implementation and its settings, like whether it supports HTTP/2. AWS publshed the decision tree which HTTP client to choose depending on the criteria.

In the next article of the series we'll make the same measurements but for the asynchronous HTTP Clients.

Update on 06.06.2024. For the CRT client we can set classifier (i.e. linux-x86_64) in our POM file to only pick the relevant binary for our platform. See here. Big thanks to Maximilian Schellhorn for the hint!