Avoid Spring RestTemplate Default Implementation to Prevent Performance Impact

Introduction

In the world of software engineering where we have so many frameworks that provide outstanding functionalities and out of box features, we tend to overlook underneath implementation and configuration. One such case is Spring RestTemplate, any APIs which are using the default RestTemplate constructor or RestTemplateBuilder's build method (no parameter) to create a RestTemplate instance will also suffer from its shortcoming.
In this post I’ll talk about the performance issues that it possesses and its solutions.

Bottleneck

Let's take a simplified example to understand the problem better.

Below is the illustration of the problem.

As depicted in above diagram, assume that we have a weather API that provides an HTTP GET endpoint to fetch data, which in-turn uses a downstream API (forecast API), let's say the weather API's response time is 200 ms (milliseconds), so with simple calculation this API could serve 5 requests per second, however, if it uses RestTemplate default implementation then maximum no. of HTTP connections per host (in this case for forecast API) is defaulted to 2 connections. So, even though API is capable of serving 5 requests per second due to a max no. of 2 HTTP connections, remaining 3 requests have to wait until a response is received from downstream API (assuming here forecast API response time is 200 ms as well). This creates a bottleneck and performance issue.

Deepdive

In Spring Framework, the RestTemplate class utilizes an underlying HttpClient implementation for handling HTTP requests. By default, RestTemplate uses the SimpleClientHttpRequestFactory, which creates a new HttpURLConnection (based on the JDK's own HTTP libraries) for each request. This factory does not have built-in connection pooling.
However, starting from Spring 4.3, the RestTemplate class can be configured to use HttpComponentsClientHttpRequestFactory, which is based on Apache HttpClient and provides connection pooling capabilities. The number of connections in the pool can be customized through the configuration.
But again, the HttpComponentsClientHttpRequestFactory does not specify the maximum number of connections in the pool. Instead, it uses the Apache HttpClient's default value, which is 2 connections per route (per target host).

Solutions

There are multiple ways to fix this problem, depending upon the level of granular customization needed for an API. We need to start with adding Apache HttpClient in the project's dependency.

Add Apache HttpClient dependency to the project

Here is an example for maven project.

<dependency>
    <groupId>org.apache.httpcomponents</groupId>
    <artifactId>httpclient</artifactId>
    <version>4.5.13</version> <!-- or the latest version -->
</dependency>

1. First Method

High level setting with minimum configuration. Setting max connection and connection per route

    @Bean
    public CloseableHttpClient httpClient() {
        return HttpClientBuilder.create()
                .setMaxConnTotal(100)      // Set required maximum total connections
                .setMaxConnPerRoute(20)    // Set required maximum connections per route
                .build();
    }

    @Bean
    public RestTemplate restTemplate() {
        return new RestTemplate(new HttpComponentsClientHttpRequestFactory(httpClient()));
    }

GitHub link to complete file
BasicRestClientCustomConnectionConfig.java

2. Second Method

More granular setting, creating more fine level configuration. Setting connection timeout, max connection and connection per route

    @Bean
    public PoolingHttpClientConnectionManager customizedPoolingHttpClientConnectionManager(){
      /*
        Setting the connection caching time to 5 minutes, so that the connection is in an open state for 5 mins. 
Every connection will add some extra time as every 5 mins SSL handshake will happen. However, software is all about trade-off, and here we are making sure, we don't keep calling cached API DNS
      */
      PoolingHttpClientConnectionManager connManager = new PoolingHttpClientConnectionManager(5, TimeUnit.MINUTES);
      connManager.setMaxTotal(100);    // Set required maximum total connections
      connManager.setDefaultMaxPerRoute(20);     // Set required maximum connections per route
      return connManager;
    }

    @Bean
    public RestTemplate restTemplate() {
        RequestConfig reqConfig = RequestConfig.custom()
                                  .setConnectionRequestTimeout(4000) //in milliseconds
                                  .setConnectionTimeout(4000)
                                  .setSocketTimeout(4000)
                                  .build();

        HttpClientBuilder clientBuilder = HttpClientBuilder.create()
                                  .setConnectionManager(customizedPoolingHttpClientConnectionManager)
                                  .setConnectionManagerShared(true)  //this is important to set as true in case of more than one downstream APIs as we want to set a common HTTP connection pool                   .setDefaultRequestConfig(reqConfig);

        HttpComponentsClientHttpRequestFactory reqFactor = new HttpComponentsClientHttpRequestFactory();
        reqFactor.setHttpClient(clientBuilder.build();

        return new RestTemplate(reqFactor);
    }

GitHub link to complete file AdvanceRestClientCustomConnectionConfig.java

Conclusion

Once you have defined max total connection and connection per route as stated above. More connections are available at the API level to make more parallel downstream API calls, which will improve the overall performance as now API can serve more requests simultaneously.
Going to our earlier example, now with more connections it would look like the below diagram.

If you have reached here, then I did a satisfactory effort to keep you reading. Please be kind to leave any comments or ask for any corrections.

My Other Blogs:

• Cracking Software Engineering Interviews
• Team Agreement in Software Engineering
• Test SOAP Web Service using Postman Tool
• My firsthand experience with web component - learnings and limitations
• Streaming logs of all pods at once
• Certified Kubernetes Application Developer (CKAD) Exam Tips