I'm working at a large Java/Kotlin project at my company. It's several 100k lines of server-side code, with millions more coming in through dependencies. The Kotlin compiler sure has its work cut out for it. But as of late, the team noticed that the compiler was getting slower and slower, even though the project source size wasn't growing nearly at the same rate. Also, builds were consuming an abundance of CPU resources - on all available cores. Build times for a full project build had increased from about 2 minutes to more than 10 minutes. Something was definitely wrong.

A couple of days ago, the whole bubble burst and it became blatantly clear what was going on:

java.lang.OutOfMemoryError: GC Overhead Limit Exceeded

... coming from a Kotlin compiler class. It turned out that it wasn't kotlinc itself which was burning our CPU - it was the JVM it was running on, struggling really hard to free memory resources.

The question obviously was: how do you assign more RAM to the kotlinc process? Easier said than done.

The Setup

Like many other folks out there, we use IntelliJ IDEA to write our source code, which is then handed to Gradle to build it. Nothing too complicated, right? Well, if we take a closer look, this is what actually happens...

Here's what each step (roughly) does:

• IntelliJ: Our IDE. Basically just invokes ./gradlew compile and monitors its output.
• gradlew: The gradle wrapper. A shell script which downloads the desired version of gradle (specified in the build), and passes on whatever arguments you gave it to this specific gradle version.
• gradle: The actual gradle executable. Here, the compileKotlin call is evaluated, and the task gets passed to the gradle daemon (which is started if it isn't already running).
• gradle daemon: A background worker created by gradle. Here the actual heavy lifting is performed.
• kotlinc: The actual kotlin compiler.

There are two very important things to realize here:

1. Gradle, IntelliJ and kotlinc are all implemented on top of the JVM.
2. Each of the processes runs in its own JVM instance. From the perspective of the operating system, they are different processes - with different memory pools.

The conclusion here is that no matter how much memory you assign to IntelliJ IDEA, it won't change the amount of RAM available to kotlinc upon build. Also, no matter how much memory you give to gradle, since kotlinc is its own process, it won't benefit from it.

The Fix

With all the insights gathered so far, the question still remains how to assign memory to the kotlinc process. For us, the following worked:

org.gradle.jvmargs=-Xmx8g -Dkotlin.daemon.jvm.options=-Xmx6g

We put this line into a file named gradle.properties, which is located right next to our gradlew executable (so, top-level in the project). This will assign 8GB of RAM to the gradle process, as well as another 6GB dedicated to the kotlin compiler.

You can of course play with the numbers. It's an implementation detail of the kotlin gradle plugin whether it calls kotlinc as a standalone process or embedded, so I took the defensive option and assigned more RAM to gradle than to kotlinc, but other configurations may work as well.

A note of caution: if you assign more RAM to a JVM via -Xmx than your OS can actually provide to it, it may fail with a segmentation fault. Never assign more RAM than you actually have.

Now, if you change those properties in the file and save it, your kotlin compiler may still be slow and/or run out of memory. That's because your gradle daemon may still be running in the background, using your previous settings. Also, IntelliJ has an abundance of caches. The surefire way to make sure your changes are actually taking effect is to:

1. Reboot your machine. I mean it. This will kill the gradle daemon, plus any in-memory caches used by IntelliJ.
2. Start IntelliJ.
3. Inside IntelliJ, do not click "build" just yet. Your gradle settings may be served from a persistent cache, which is clearly not what we want. Instead, refresh your gradle project first.
4. Once the refresh is completed, perform a full project rebuild from IntelliJ.

The results

Our builds are now a lot faster again! We went from over 10 minutes at 100% CPU back to less than 1 minute, with much more reasonable CPU loads. Because the change was done in the build setup, our CI/CD server builds are now faster as well.

Note: This procedure works for Kotlin/JVM for a server-side project built by gradle. It may or may not work for Android builds, Kotlin/JS, Kotlin/Native or multiplatform projects.

Conclusion

If your Kotlin compiles take way longer than they should, and your CPU goes crazy on all cores while kotlin is compiling, consider increasing the maximum heap size of your kotlin compiler. It may be running low on memory.