Java and Docker - Memory and CPU Limits

This article originated on zsiegel.com

For many years java developers have gotten used to tweaking their code and the JVM flags to gain both performance and stability.

Now that we are deploying more often to containers its important to also understand how the underlying container technologies effect our application.

Let’s take a look at a real world example to understand how our application reacts when we adjust container flags that control memory and CPU allocation.

This example is applicable to Docker, Kubernetes, Mesos and other orchestrated container environments.

All of the following commands can be run on Docker CE for Linux and Mac v18 and up. We use JDK8 in these examples to demonstrate the lowest container aware JVM.

A Simple Executable

Let’s start off with a simple Java executable that logs to the console the processors detected and the max memory available to the runtime.

public class Docker {

  public static void main(String[] args) {

    Runtime runtime = Runtime.getRuntime();

    int processors = runtime.availableProcessors();
    long maxMemory = runtime.maxMemory();

    System.out.format("Number of processors: %d\n", processors);
    System.out.format("Max memory: %d bytes\n", maxMemory);
  }
}

We then create a simple Dockerfile that contains the JAR with this main function.

FROM openjdk:8-jre
COPY /build/libs/java-and-docker-1.0.jar java-and-docker.jar

CMD ["java", "-XX:+UnlockExperimentalVMOptions", "-XX:+UseCGroupMemoryLimitForHeap", "-jar", "java-and-docker.jar"]

In this example, we are going to run JDK 8, which is the lowest container-aware JDK. For more information about the differences in container awareness capabilities between JDK versions, you can check out the previous article here.

Now we need to build the container.

$ docker build -t zsiegel:java-and-docker .

With the container built, let’s run this on the current machine and see what we get.

$ docker run zsiegel:java-and-docker
Number of processors: 4
Max memory: 466092032 bytes

You can compare this result to the CPU and memory settings in the Docker app preferences on the Mac or look at how it compares to your machine specs on Linux. The number of cores should match and the ram displayed should be slightly lower.

Memory Limits

Let’s take a look at limiting the memory the container can use and see how the runtime adjusts.

$ docker run -it --memory=512m zsiegel:java-and-docker
Number of processors: 4
Max memory: 119537664 bytes

Now that we have adjusted the amount of memory available to the container itself the heap size is adjusted by the runtime. How did the runtime decide on this value of 119537664 bytes which equates to roughly 120 megabytes? If you dig into the JVM Tuning guide you will see the following.

"Unless the initial and maximum heap sizes are specified on the command line, they're calculated based on the amount of memory on the machine. The default maximum heap size is one-fourth of the physical memory while the initial heap size is 1/64th of physical memory. The maximum amount of space allocated to the young generation is one third of the total heap size."

The runtime by default will use 1/4th of the available memory. In our case this means 512/4 = 128 megabytes which is roughly the number we see.

Processor Limits

Let’s take a look at limiting the cpu that is available to the container and see what happens. As of JDK 8 Update 131, the runtime should be aware of the number of cpus available and will tune thread counts accordingly. In the JVM's case one cpu is equal to one core.

$ docker run -it --cpus=1 zsiegel:java-and-docker
Number of processors: 4
Max memory: 468189184 bytes

This is not what I expected the first time I ran it so let’s dig in further to understand what is really going on.

The Docker --cpus flag specifies the percentage of available CPU resources a container can use. Specifically it refers to the cpu-shares. It does not adjust the number of physical processors the container has access to, which is what the jdk8 runtime currently uses when setting the number of processors.

There is future work to correct this shortcoming in JDK 10. You can follow the progress and discussion here

Let’s try using a different flag called --cpuset-cpus. This flag is used to limit the cores a container can use. On a 4 core machine we can specify 0-3. We can specify a single core or even multiple cores by comma separating the index of the cores.

$ docker run -it --cpuset-cpus=0 zsiegel:java-and-docker
Number of processors: 1
Max memory: 508887040 bytes 

$ docker run -it --cpuset-cpus=0,1 zsiegel:java-and-docker
Number of processors: 2
Max memory: 508887040 bytes 

This result is more in line with what we expected. We now have the runtime properly seeing 2 cores available instead of 4. When we do this the runtime will then tune the number of compiler and garbage collection threads accordingly.

It is important to note that many libraries and frameworks that rely on thread pools will tune the number of threads based on these numbers. You would think that this would address our problems but there is a catch!

Container Orchestration

There is a major problem with the above example. The vast majority of container orchestration tools like Mesos and Kubernetes set the cpu-shares and not the cpuset-cpus. This means that until the work in JDK10 mentioned earlier is completed the runtime and frameworks that rely on runtime.availableProcessors() will be unable to tune their thread count properly. My hope is that this work will be backported to at least JDK9 if possible.

If you want more info on the flags available in Docker for adjusting resource limits you can check out the documentation here