AWS SnapStart - Part 22 Measuring cold and warm starts with Java 17 using synchronous HTTP clients

Introduction

In the previous parts we've done many measurements with AWS Lambda using Java 17 runtime with and without using AWS SnapStart and additionally using SnapStart and priming DynamoDB invocation :

• cold starts using different deployment artifact sizes
• cold starts and deployment time using different Lambda memory settings
• warm starts using different Lambda memory settings
• cold and warm starts using different compilation options

In this article we'll now add another dimension to our Java 17 measurements : the choice of HTTP Client implementation. Starting from AWS SDK for Java version 2.22 AWS added support for their own implementation of the synchronous CRT HTTP Client. The asynchronous CRT HTTP client has been generally available since February 2023. In this article we'll explore synchronous HTTP clients first and leave asynchronous ones for the next article.

I will also compare it with the same measurements for Java 21 already performed in the article Measuring cold and warm starts with Java 21 using different synchronous HTTP clients

Measuring cold and warm starts with Java 17 using synchronous HTTP clients

In our experiment we'll re-use the application introduced in part 8 for this. Here is the code for the sample application. There are basically 2 Lambda functions which both respond to the API Gateway requests and retrieve product by id received from the API Gateway from DynamoDB. One Lambda function GetProductByIdWithPureJava17Lambda can be used with and without SnapStart and the second one GetProductByIdWithPureJava17LambdaAndPriming uses SnapStart and DynamoDB request invocation priming.

As we did our measurements for Java 17 in the previous articles of the series, we have always used the default HTTP Client implementation which is Apache HTTP Client (we'll use the measurements for the comparison in this article), now we'll explore 2 other options as well.

There are now 3 synchronous HTTP Clients implementations available in the AWS SDK for Java.

1. Url Connection
2. Apache (Default)
3. AWS CRT

This is the order for the look up and set of synchronous HTTP Client in the classpath.

Let's figure out how to configure the HTTP Client. There are 2 places to do it : pom.xml and DynamoProductDao

Let's consider 3 scenarios:

Scenario 1) Url Connection HTTP Client. Its configuration looks like this:
In the pom.xml the only enabled HTTP Client dependency has to be:

     <dependency>
        <groupId>software.amazon.awssdk</groupId>
        <artifactId>url-connection-client</artifactId>
     </dependency>

In DynamoProductDao the DynamoDBClient should be created like this:

DynamoDbClient.builder()
    .region(Region.EU_CENTRAL_1)
     .httpClient(UrlConnectionHttpClient.create())
    .overrideConfiguration(ClientOverrideConfiguration.builder()
      .build())
    .build();

Scenario 2) Apache HTTP Client. Its configuration looks like this:
In the pom.xml the only enabled HTTP Client dependency has to be:

     <dependency>
        <groupId>software.amazon.awssdk</groupId>
        <artifactId>apache-client</artifactId>
     </dependency>

In DynamoProductDao the DynamoDBClient should be created like this:

DynamoDbClient.builder()
    .region(Region.EU_CENTRAL_1)
     .httpClient(ApacheHttpClient.create())
    .overrideConfiguration(ClientOverrideConfiguration.builder()
      .build())
    .build();

Scenario 3) AWS CRT synchronous HTTP Client. Its configuration looks like this:
In the pom.xml the only enabled HTTP Client dependency has to be:

     <dependency>
        <groupId>software.amazon.awssdk</groupId>
        <artifactId>aws-crt-client</artifactId>
     </dependency>

In DynamoProductDao the DynamoDBClient should be created like this:

DynamoDbClient.builder()
    .region(Region.EU_CENTRAL_1)
     .httpClient(AwsCrtHttpClient.create())
    .overrideConfiguration(ClientOverrideConfiguration.builder()
      .build())
    .build();

For the sake of simplicity, we create all HTTP Clients with their default settings. Of course, there is the optimization potential there to figure out the right HTTP Client settings.

The results of the experiment below were based on reproducing more than 100 cold and approximately 100.000 warm starts with experiment which ran for approximately 1 hour. For it (and experiments from my previous article) I used the load test tool hey, but you can use whatever tool you want, like Serverless-artillery or Postman. I ran all these experiments for all 3 scenarios using 2 different compilation options in the AWS SAM template.yaml each:

1. no options (tiered compilation will take place)
2. JAVA_TOOL_OPTIONS: "-XX:+TieredCompilation -XX:TieredStopAtLevel=1" (client compilation without profiling)

We found out in the article Measuring cold and warm starts with Java 17 using different compilation options that with them both we've got the lowest cold and warm start times. We’ve also got good results with "-XX:+TieredCompilation -XX:TieredStopAtLevel=2” compilation option but I haven’t done any measurement with this option yet.

Let's look into the results of our measurements.

Cold (c) and warm (m) start time with compilation option "tiered compilation" without SnapStart enabled in ms:

Cold (c) and warm (m) start time with compilation option "-XX:+TieredCompilation -XX:TieredStopAtLevel=1" (client compilation without profiling) without SnapStart enabled in ms:

Cold (c) and warm (m) start time with compilation option "tiered compilation" with SnapStart enabled without Priming in ms:

Cold (c) and warm (m) start time with compilation option "-XX:+TieredCompilation -XX:TieredStopAtLevel=1" (client compilation without profiling) with SnapStart enabled without Priming in ms:

Cold (c) and warm (m) start time with compilation option "tiered compilation" with SnapStart enabled and with DynamoDB invocation Priming in ms:

Cold (c) and warm start (m) time with compilation option "-XX:+TieredCompilation -XX:TieredStopAtLevel=1" (client compilation without profiling) with SnapStart enabled and with DynamoDB invocation Priming in ms:

Conclusion

In terms of the HTTP Client choice for Java 17, AWS CRT HTTP Client is preferred choice in case SnapStart isn't enabled or SnapStart is enabled but no priming is applied. In case of priming of the DynamoDB invocation, the results in terms of the cold starts for all 3 HTTP Clients are close to each other as the initialization of the DynamoDB Client with the HTTP Client and the most expensive first invocation (priming) happens already during the deployment phase of the Lambda function and doesn't impact the further invocations that much. The Apache HTTP Client is probably the most powerful choice, but it shows the worst results for the cold starts for SnapStart not being enabled.

We observed the same also for Java 21, see the measurements in the article Measuring cold and warm starts with Java 21 using different synchronous HTTP clients.

The warm execution times are more or less close to each other for all 3 HTTP clients and compilation options and very a bit in favor of one or another HTTP Clients depending on the percentile and compilation option. We observed the same also for Java 21

Can we reduce the cold start a bit further? From our article Measuring cold starts with Java 17 using different deployment artifact sizes we know that smaller deployment artifact sizes lead to the lower cold start times. The usage of AWS CRT HTTP Client adds 18 MB to the deployment artifact size for our sample application (total size 32MB versus 14 MB for URL Connection and Apache HTTP Clients). If we look into the deployment artifact with AWS CRT HTTP Client, we'll discover the following additional packages for each operating system : linux, osx and windows.

If we take a look into those folders, we'll see for example the following content for the linux folder (the same applies for windows and osx folders) :

As we see the content of such folders is natives file for each operating system and processor architecture: for osx it's libaws-crt-jni.dylib file, for windows - aws-crt-jni.dll and for linux - libaws-crt-jni.so. If we already know that we'll run our Lambda only on Linux x86 architecture, we can delete the osx and windows folders completely and subfolders for arm architecture in the linux folder. This will reduce the deployment artifact size from 32 to 19 MB for AWS CRT HTTP Client and further reduce the cold start time a bit.

The choice of HTTP Client is not only about minimizing cold and warm starts. The decision is much more complex end also depends on the functionality of the HTTP Client implementation and its settings, like whether it supports HTTP/2. AWS publshed the decision tree which HTTP client to choose depending on the criteria.

In the next article of the series we'll make the same measurements for Java 17 but using the asynchronous HTTP Clients.

Update on 06.06.2024. For the CRT client we can set classifier (i.e. linux-x86_64) in our POM file to only pick the relevant binary for our platform. See here. Big thanks to Maximilian Schellhorn for the hint!