Going Native: Trying Out AOT For Spring Boot

Background

One of the key selling points for the Java programming language, when it was released, was the promise of “write once, run anywhere”. That is to say, that code written in Java would not need to be compiled into native code specific to an exact computer architecture. Instead, the Java code would be compiled into “byte code”, i.e. instructions that would be interpreted and executed by the Java Virtual Machine (JVM) that would be installed on the host computer. This provided the freedom in the software development process for Java developers – no need to maintain build machines for Linux, MacOS, Windows, etc – at the cost of reduced performance due to several factors, among others:

• The intermediate step of the JVM interpreting and executing the compiled byte code.
• The JVM would itself need to initialize and begin to load/execute the code, i.e. a “cold start” that would add delay before the program could be executed.

In response, Oracle produced the GraalVM, one of the features of which is the ability to compile JVM-compatible code into a “native” executable and reduce the start-up time of such a program drastically. A key constraint to this is that the JVM program is transformed from using “Just-In-Time” (JOT) compilation to “Ahead-Of-Time” (AOT) compilation, meaning that there is no mechanism to load and modify a program’s class structure at runtime when it is compiled into an executable. While the GraalVM employs some limited methods to convert instances of reflective programming into AOT-compatible code, in many cases, the developer will need to provide configuration “hints” to the GraalVM compiler in the form of configuration files. This would be an annoyance for JVM programs leveraging the Spring Boot framework, as Spring Boot heavily leverages reflection and dynamic class loading to provide benefits like dependency injection and generation of class delegates for a variety of objectives.

Thankfully, this situation has begun to change. The developers of Spring Boot have released version 3.0 of the widely-used framework, and one of the “star” attractions of the new version is the refactoring and development that has been done to make a Spring Boot program compatible with GraalVM. This would theoretically allow the best of both worlds: a program that leverages the autowiring of dependencies via dependency injection and initiates quickly as well. Curious about how this works out, I created a sample program that I would attempt to get up and running as a compiled executable; any modifications to the program would be conducted in a separate branch to provide an A/B contrast of what code works for either paradigm.

The Work

Setup Notes

The sample program is a Gradle-based Spring Boot WebFlux application that is written in Kotlin and leverages the following features:

• REST controllers
• Kotlin Coroutines and suspending functions
• Kotlin-based serialization
• JPA and Hibernate
• A database layer (H2)
• Aspect-Oriented Programming (AOP) for logging how much time elapses in each web request
• Login security
• Profile-based bean loading for using or disabling the security mechanism

When running this application and conducting three consecutive GET requests to the authors/1endpoint, the following log output is produced:

Started BootImageDemoKt in 4.691 seconds (process running for 5.263)
Time elapsed for getAuthor: 15095 µs
Time elapsed for getAuthor: 435 µs
Time elapsed for getAuthor: 166 µs

So far, quite typical for a Spring Boot program. As mentioned above, the cold start of the program means that several seconds will elapse before the web application will be able to respond to the first request, something that precludes this type of program from use in an environment where fast responses are required for sporadic requests, e.g. in an AWS lambda-based microservices application.

In any case, a Docker-based environment should be up and running when conducting the executable compilation, as the end result will be an image containing the executable that can subsequently be run using Docker (or Podman) commands; more information about the specific build command(s) can be found here. In addition, the development work was conducted on a MacOS and uses Podman instead of the traditional Docker engine for running containers. This should not cause issues, but additional configuration might be required, as I’ll describe below.

Issue #0: Gradle Configuration

First and foremost, the GraalVM native building plugin must be installed in the Gradle build file:

plugins {
   kotlin("jvm") version "1.7.21"
   kotlin("plugin.allopen") version "1.7.21"
   kotlin("plugin.jpa") version "1.7.21"
   kotlin("plugin.serialization") version "1.7.21"
   id("org.springframework.boot") version "3.0.0"
   id("io.spring.dependency-management") version "1.0.11.RELEASE"
   id("org.graalvm.buildtools.native") version "0.9.18" // <= Add this
}

This will provide the tasks aotClasses and aotTestClasses – for production and test classes, respectively – that will conduct the generation of code for various Spring components that the AOT compiler requires.

Issue #1: Task Configuration

Executing the Gradle task bootBuildImage for the first time quickly runs into an error:

Unable to parse name "BootImageDemo". Image name must be in the form '[domainHost:port/][path/]name', with 'path' and 'name' containing only [a-z0-9][.][_][-]

This is a very easy issue to fix via specifying what the compiled image will be named in the Gradle task configuration, in this case changing to using snake case instead of camel case for the project’s image name:

tasks {
   named<BootBuildImage>("bootBuildImage") {
       imageName.set("boot_image_demo")
   }
   // Other tasks omitted
}

A note: if running Podman, there might be issues with the Gradle task connecting to the virtual machine’s socket. The address should be unix:///var/run/docker.sockand reachable by default, but it may be necessary to configure it in the Gradle task configuration as well:

tasks {
   named<BootBuildImage>("bootBuildImage") {
       imageName.set("boot_image_demo")
       docker {
           host.set("unix:///var/run/docker.sock")
           bindHostToBuilder.set(true)
       }
   }
   // Other tasks omitted
}

Issue #2: Virtual Machine Configuration

Once the output image is properly configured, the next errors encountered are related to the setup of the virtual machine that hosts the GraalVM conducting the compilation, the first of which was:

Docker API call to 'localhost/v1.24/containers/create' failed with status code 500 "Internal Server Error" and message "container create: statfs /var/run/docker.sock: permission denied"

This error is due to how Podman instantiates virtual machines. Conforming to the principle of least privilege, Podman does not launch virtual machines with root privileges by default, as evidenced below:

% podman machine start
Starting machine "podman-machine-default"
Waiting for VM ...
Mounting volume... /Users/severneverett:/Users/severneverett

This machine is currently configured in rootless mode. If your containers
require root permissions (e.g. ports < 1024), or if you run into compatibility
issues with non-podman clients, you can switch using the following command: 

    podman machine set --rootful

API forwarding listening on: /var/run/docker.sock
Docker API clients default to this address. You do not need to set DOCKER_HOST.

However, the bootBuildImage task will require root privilege in order to conduct its operations, so the virtual machine needs to be configured as such:

% podman machine stop                            
Waiting for VM to stop running...
Machine "podman-machine-default" stopped successfully
% podman machine set --rootful=true
% podman machine start             
Starting machine "podman-machine-default"
Waiting for VM ...
Mounting volume... /Users/severneverett:/Users/severneverett
API forwarding listening on: /var/run/docker.sock
Docker API clients default to this address. You do not need to set DOCKER_HOST.

Machine "podman-machine-default" started successfully

Next, the image compilation process eventually fails due to the following error:

    [creator] Error: Image build request failed with exit status 137

Again, this is an issue with Podman’s defaults. Without specifying otherwise, virtual machines created in Podman on my machine possess 2GB of RAM, whereas it’s recommended that the virtual machine have at least 8GB of RAM. Thus, I had to create a new virtual machine with 8GB of RAM and 2 CPUs, after which the compilation proceeded to finish without any more error messages… after at least 10 minutes. Quite a long time to wait for a compiled image, but we’ll see how the performance of the program benefits from this compilation.

Issue #3: Kotlin Hints

With the compiled image now ready, it’s possible to create a container for the image using the command docker run -p 8080:8080 docker.io/library/boot_image_demo, after which we see the first results of the efforts:

Started BootImageDemoKt in 0.233 seconds (process running for 0.238)

*Much* faster than traditionally running the program! Unfortunately, the success is still fleeting at this point. Running the same authors/1 GET request now hangs; within the container’s logs sits this snippet:

Caused by: java.lang.NoSuchMethodException: kotlin.internal.jdk8.JDK8PlatformImplementations.<init>()
        at java.base@17.0.5/java.lang.Class.getConstructor0(DynamicHub.java:3585) ~[com.severett.bootimagedemo.BootImageDemoKt:na]
        at java.base@17.0.5/java.lang.Class.newInstance(DynamicHub.java:626) ~[com.severett.bootimagedemo.BootImageDemoKt:na]
        ... 203 common frames omitted

The good news is that this is a known issue, and the workaround is straightforward: create the file /src/main/resources/META-INF/native-image/reflect-config.json and populate it with the content outlined here.

Issue #4: Replacing Profile-Based Loading

Rebuilding the image with the new configuration file now produces an image that does not crash when web requests are made, so now we turn our attention to parts of the original program that did not get included in the native image compilation. To understand the first of these issues, it’s necessary to briefly delve into the way that a Spring Boot application is adapted to AOT compilation. As previously mentioned, Spring Boot applications leverage dependency injection to construct and populate objects in the system without the developer having to write this wiring code out themselves. It’s a quite involved topic, but the short of it is that objects that are designated as “beans” – either via a class annotated with an annotation that inherits from Spring’s @Component annotation or as an instance returned in a @Bean-annotated function within a @Configuration-annotated class – are gathered, instantiated, and supplied wherever needed within the application (usually as an autowired dependency in another bean class).

As this bean registration and autowiring is conducted reflectively in Spring, it is incompatible with the AOT compilation system of GraalVM. Instead, the developers of Spring Boot constructed a mechanism that generates the code necessary to execute this auto-wiring programmatically instead of using reflection. For instance, here are the classes that Spring Boot generates for the executable image:

Here is what is generated within one of the BeanDefinitions classes, specifically, the class generated for KotlinSerializationConfiguration:

public class KotlinSerializationConfiguration__BeanDefinitions {
   public KotlinSerializationConfiguration__BeanDefinitions() {
   }

   public static BeanDefinition getKotlinSerializationConfigurationBeanDefinition() {
       Class<?> beanType = KotlinSerializationConfiguration.class;
       RootBeanDefinition beanDefinition = new RootBeanDefinition(beanType);
       ConfigurationClassUtils.initializeConfigurationClass(KotlinSerializationConfiguration.class);
       beanDefinition.setInstanceSupplier(KotlinSerializationConfiguration..SpringCGLIB..0::new);
       return beanDefinition;
   }

   private static BeanInstanceSupplier<KotlinSerializationJsonHttpMessageConverter> getConfigBeanInstanceSupplier() {
       return BeanInstanceSupplier.forFactoryMethod(KotlinSerializationConfiguration.class, "configBean", new Class[0]).withGenerator((registeredBean) -> {
           return ((KotlinSerializationConfiguration)registeredBean.getBeanFactory().getBean(KotlinSerializationConfiguration.class)).configBean();
       });
   }

   public static BeanDefinition getConfigBeanBeanDefinition() {
       Class<?> beanType = KotlinSerializationJsonHttpMessageConverter.class;
       RootBeanDefinition beanDefinition = new RootBeanDefinition(beanType);
       beanDefinition.setInstanceSupplier(getConfigBeanInstanceSupplier());
       return beanDefinition;
   }
}

Lastly, here is how the bean declared within KotlinSerializationConfiguration gets registered within for the AOT-compatible bean definition system in BootImageDemoKt__BeanFactoryRegistrations::registerBeanDefinitions():

public void registerBeanDefinitions(DefaultListableBeanFactory beanFactory) {
   /* Other registrations */
   beanFactory.registerBeanDefinition("kotlinSerializationConfiguration", KotlinSerializationConfiguration__BeanDefinitions.getKotlinSerializationConfigurationBeanDefinition());
   /* Other registrations */
   beanFactory.registerBeanDefinition("configBean", KotlinSerializationConfiguration__BeanDefinitions.getConfigBeanBeanDefinition());
   /* Other registrations */
}

All is well and good, right? Wrong. Take a look again at the generated classes within the config package: you’ll notice that there are no generated classes for the security configuration beans. This is because the dynamic bean loading is incompatible with the AOT compilation system, so any beans that employ the @Profile annotation, the @ConditionalOnPropertyannotation, etc. are not permitted and thus do not appear within the native executable. The workaround, in this case, is to migrate the conditional bean loading logic to reside within the function that generates the Spring bean:

private const val SECURITY_ENABLED = "bootimagedemo.securityEnabled"

@Configuration
class SecurityConfiguration(private val env: Environment) {
   @Bean
   fun springSecurityFilterChain(http: ServerHttpSecurity): SecurityWebFilterChain {
       val securityEnabled = env[SECURITY_ENABLED]?.toBoolean() ?: false
       return if (securityEnabled) {
           http.authorizeExchange { exchanges ->
               exchanges.anyExchange().authenticated()
           }
               .httpBasic(withDefaults())
               .build()
       } else {
           http.authorizeExchange { exchanges ->
               exchanges.anyExchange().permitAll()
           }
               .csrf().disable()
               .build()
       }
   }
}

The other refactoring is how to trigger the various security configurations: instead of passing in a profile for loading, it will now be necessary to pass in the bootimagedemo.securityEnabledproperty, either in the configuration file or via a command-line argument.

Issue #5: Fixing AOP

Lastly, the compiled image conducts the web requests correctly, but one part is still missing: the log output from the AOP function that measures the time that elapses in each controller function. Just like the previous issue, this one requires a bit of background explanation before diving in. AOP in Spring functions by inserting code around a targeted function – designated either by a specific annotation or by matching a string pattern (as is done in this project) – which is to be executed before the targeted function, after it, or both. Spring handles this by generating a pass-through class on top of the class that contains the targeted function; when the targeted function is called, the pass-through class checks for whether a callback function for the targeted function is present and calls that instead of the targeted function (the callback should theoretically subsequently call the targeted function). These classes are generated at runtime in a normal Spring Boot application, but the AOT compilation generates the classes beforehand. Thus, as both AuthorControllerand BookController contain functions that are affected by the AOP pattern-matching, pass-through classes are generated for them as can be seen here:

Note the other classes that have pass-through classes generated for them; these are due to the same mechanism being necessary for classes annotated with @Configuration or @Transactional, among others. So, what is causing the AOP code to not function correctly in the native executable? This is due to a pair of issues. First, there is a bug that requires additional runtime hints for the AOT-compiled program to recognize the AOP code correctly:

class AspectRuntimeHints : RuntimeHintsRegistrar {
   override fun registerHints(hints: RuntimeHints, classLoader: ClassLoader?) {
       hints.reflection().registerType(TimingAspect::class.java) { builder ->
           builder.withMembers(MemberCategory.INVOKE_DECLARED_METHODS)
       }
       hints.proxies().registerJdkProxy(
           FactoryBean::class.java,
           BeanClassLoaderAware::class.java,
           ApplicationListener::class.java
       )
       hints.proxies().registerJdkProxy(ApplicationAvailability::class.java, ApplicationListener::class.java)
   }
}

In addition, the annotation @ImportRuntimeHints needs to be added to the class that contains the AOP code:

@Aspect
@ImportRuntimeHints(AspectRuntimeHints::class)
@Component
class TimingAspect {

With this in place, we come up to the second issue wherein the AOT-compiled image starts up… and promptly crashes:

2022-12-01T16:26:34.556Z WARN 1 --- [main] .r.c.ReactiveWebServerApplicationContext : Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'entityManagerFactory': Unsatisfied dependency expressed through method 'entityManagerFactory' parameter 0: Error creating bean with name 'entityManagerFactoryBuilder': Unsatisfied dependency expressed through method 'entityManagerFactoryBuilder' parameter 0: Error creating bean with name 'org.springframework.boot.autoconfigure.orm.jpa.HibernateJpaConfiguration': Unsatisfied dependency expressed through constructor parameter 0: Error creating bean with name 'dataSource': Proxy class defined by interfaces [interface javax.sql.DataSource, interface java.io.Closeable, interface com.zaxxer.hikari.HikariConfigMXBean] not found. Generating proxy classes at runtime is not supported. Proxy classes need to be defined at image build time by specifying the list of interfaces that they implement. To define proxy classes use -H:DynamicProxyConfigurationFiles=<comma-separated-config-files> and -H:DynamicProxyConfigurationResources=<comma-separated-config-resources> options.

This is, thankfully, not a complicated problem to fix in the demo code. There is a separate bug where “complicated” pattern matching for AOP causes crashes in the compiled Spring Boot application as seen above. The original AOP pattern in the demo application essentially wraps any public function in any controller:

private val logger = KotlinLogging.logger { }

@Aspect
@ImportRuntimeHints(AspectRuntimeHints::class)
@Component
class TimingAspect {
   @Around(value= "execution(* com.severett.bootimagedemo.controller.*Controller.*(..))")
   fun aroundAdvice(pjp: ProceedingJoinPoint): Any {
       val startTime = getNanoTime()
       try {
           return pjp.proceed()
       } finally {
           val endTime = getNanoTime()
           if (startTime != null && endTime != null) {
               logger.debug { "Time elapsed for ${pjp.signature.name}: ${(endTime - startTime) / 1_000} µs" }
           }
       }
   }

   private fun getNanoTime() = if (logger.isDebugEnabled) System.nanoTime() else null
}

Thus, the workaround would be to eliminate the *Controllerwildcard matching and create pass-through functions that specifically match the functions of both controllers:

@Aspect
@ImportRuntimeHints(AspectRuntimeHints::class)
@Component
class TimingAspect {
   @Around(value= "execution(* com.severett.bootimagedemo.controller.AuthorController.*(..))")
   fun aroundAuthorController(pjp: ProceedingJoinPoint) = execTimingAdvice(pjp)

   @Around(value= "execution(* com.severett.bootimagedemo.controller.BookController.*(..))")
   fun aroundBookController(pjp: ProceedingJoinPoint) = execTimingAdvice(pjp)

   private fun execTimingAdvice(pjp: ProceedingJoinPoint): Any {
       val startTime = getNanoTime()
       try {
           return pjp.proceed()
       } finally {
           val endTime = getNanoTime()
           if (startTime != null && endTime != null) {
               logger.debug { "Time elapsed for ${pjp.signature.name}: ${(endTime - startTime) / 1_000} µs" }
           }
       }
   }

   private fun getNanoTime() = if (logger.isDebugEnabled) System.nanoTime() else null
}

End Result

Finally, the code compiles completely and produces the output that we want:

Started BootImageDemoKt in 0.21 seconds (process running for 0.226)
Time elapsed for getAuthor: 815 µs
Time elapsed for getAuthor: 68 µs
Time elapsed for getAuthor: 68 µs

As the results show, not only is the start-up of the program much faster, but the initial GET calls to /author/1take much less time to complete as well.

Final Thoughts

While the basic performance test showed a marked improvement between the traditional Spring Boot application and the AOT-compiled application, I would be hesitant to recommend converting every Spring Boot application into its AOT equivalent, at least at this point.

• Given that dynamic program execution is not possible with an AOT-compiled application, several key strengths of both Spring Boot and the JVM – namely, the profile- and configuration-dependent class loading and the JIT compilation optimizations, respectively – have to be sacrificed. It may be possible that the Spring Boot developers will figure out a way to enable profiles and the like in the AOT-compiled code, although – like C++ templating code – it may come at the cost of additional code generation. Nevertheless, this still leaves the JIT optimizations that would be impossible in an AOT-compiled program, meaning no optimization of code hotspots (i.e. frequently-run code).
• The hardware and time requirements for building a native executable are substantial. As stated above, the GraalVM compiler required a virtual machine with at least 8GB of RAM on my computer – meaning half of my computer’s memory just for the native image compilation – and took at least ten minutes to build the native executable for what is, essentially, a trivial program. Aside from the costs of having to maintain a build process that fulfills these hardware requirements, there’s also the dilemma of what tradeoffs between which programs to run – or even whether to purchase a more powerful computer – that a developer might need to consider when working on and compiling a larger program.
• The proscribed development workflow is not reliable. As mentioned above, the Gradle tasks aotClasses and aotTestClasses generate the Spring classes that are required for the AOT compilation. With these generated classes, it is theoretically possible to run the Spring Boot program in “AOT mode” wherein the program will be run as a traditional JVM application, yet the classes will be loaded in the style of the AOT-compiled equivalent, i.e. loading the Spring beans via the generated code instead of dynamically. The objective of this is precisely to avoid the long compilation process, yet the issues with the AOP code raised some serious concerns:
  • As the first and second AOP-related bugs both describe, the code functioned perfectly well in this “AOT mode”, yet when the actual compiled code ran, the actual issues arose. This means that there’s no actual guarantee that a developer will be able to avoid this expensive compilation step when working on an AOT-compiled project.
  • The second bug showed that the error-reporting mechanism for AOT-compiled code needs refinement. Although the error message ostensibly indicated that the problematic code was situated in the Spring JPA library, it was ultimately illusory, as removing the JPA code simply meant that the Spring Boot application would crash and print out an error message on a different part of the code.

Despite this, it’s encouraging to see that there is serious progress being made to address what has perennially been one of the JVM ecosystem’s weaknesses. This is the first release of the AOT compilation mechanism for Spring Boot – and the very AOT compilation process within GraalVM is also relatively young – so there is likely much room for improvement in the future. Java itself faced much criticism in its beginning for being very performance-deficient in comparison to languages like C++, yet its evolution progressed such that a major video game written in Java (i.e. Minecraft) was eventually produced, something that would’ve been inconceivable at Java’s inception. Time will tell if such a level of improvement will be in store for GraalVM and Spring Boot in the future.

If you enjoyed this article, I suggest you keep an eye on Apiumhub’s blog. New and exciting content on backend development, frontend development, software architecture, DevOps, QA automation, and more gets published every week.