Java memory model

The goal of this post is to get familiar with how memory works in java. We will see how it works in combination with the OS it runs on, how it's structured and how it functions internally inside a JVM.

This post is a product of my research on the topic using freely available resources that you can also find on the internet.

What we all probably know is that java applications are executed inside a JVM. JVM uses a memory from the OS it's running on.

This memory inside a JVM can now be split into two categories:

• Heap
• Non-heap

Heap

Heap memory is used for dynamic allocation of java object and java classes in runtime. Newly created objects are always saved into heap space while the references to these objects are always in stack memory.

These objects have global access, which means we can access them from anywhere inside our application.

Basic features of heap memory are:

• Access to this memory is done through complex techniques and is comparably a lot slower than accessing stack memory
• If heap is full Java will throw Java.lang.OutOfMemoryError
• This memory isn't automatically deallocated, it needs a garbage collector to clean up objects no longer in use
• Unlike stack, heap memory isn't thread safe and needs to be properly synchronized

Stack

Stack memory is used for static allocation and thread execution. It uses primitive values that are specific to the method being executed and references to objects used it this method.
Access to this memory is LIFO. Whenever we invoke a new method, a block is created at the top of the stack which holds values specific to this method. When the method is done executing this block is destroyed and the memory is free again.

Basic features of this memory are:

• It grows and shrinks as new methods are being invoked
• Variables inside stack exists only as long as the method they're being used in is executing
• It's automatically allocated and deallocated as methods are being executed
• If this memory is full java will throw java.lang.StackOverflowError
• Access to this memory is fast compared to heap
• This memory is thread safe, each thread has it's own stack

Metaspace

Metaspace is a special kind of heap memory separate from the main heap memory. JVM stores classes metadata in Metaspace. Also all the static content is saved in metaspace, which includes all static methods, primitive variables and references to static objects. It also includes data about bytecode and JIT information's.
Prior to Java 7, string pool was also part of this memory.
It was also called Permament generation prior to Java 7, and the main difference between perm gen and metaspace is how memory allocation is handled.
Specifically in metaspace this memory grows automatically by default.

Code cache

Code cache is a space inside JVM where the compiled bytecode is stored. The biggest consumer of this memory is obviously the JIT compiler so you might also hear this memory referred as to JIT code cache.

How does JVM manage heap

When JVM starts up, it requires a memory for the heap. Heap is as we already mentioned a memory where all the objects will be stored, and they'll be referenced from the stacks.
Initial size of the heap can be configured with **-Xms** argument. JVM will dynamically allocate memory from the heap up to the maximum heap size which can also be configured with the **-Xmx** argument.
If we do not specify these arguments JVM will automatically set initial and maximum size of heap based on the host's capacities.
If heap reaches maximum size and requires more memory we will get java.lang.OutOfMemory error

We already told that this method is dynamically allocated, but how is it deallocated?
As the application is creating new objects, JVM will automatically allocate memory from the heap and heap usage will grow. JVM will also periodically run GC (garbage collection) to free the space from objects which are no longer used (no longer referenced). This ensures that the JVM has enough memory to keep allocating newly created objects.

GC in more detail

GC is necessary for freeing up memory, but it also pauses all application threads which can lead to users problems in latency.
Ideally JVM should run GC often enough to free enough memory that the application needs, but not too often to interrupt application activity unnecessarily.

GC algorithms have evolved during the years to allow minimum pauses and to free up as efficiently as possible. Since Java 9 Garbage-First GC (short G1 GC) is the default GC. G1 GC is currently the best option for production applications that require large amounts of heap memory and short pauses in application activity. So now we will focus a bit more on G1 GC.

G1 splits the heap in equal regions, so that we have "young generation" and "old generation". Young generation is consited of eden and survivor regions, while the old generation consists of humongous regions (humongous for storing large objects, objects that take up more than 50% of available memory).

With the exception of these humongous objects, all newly created objects are initially stored into eden space of young generation, and then they are moved to the survivor regions based on how many GC cycles they survive.

But what does it mean to survive a GC cylcle?

Phases of GC

We have 3 types of collection that GC does:

• Young only
• Mixed collection
• Full gc

G1 GC collection cycles alternate between young only and space reclamation phase.

During the young only phase, G1 collector does two types of processes:

• Young garbage collection, which involves evacuation of living objects from the eden to survivor regions
• Marking cycle, which includes taking inventory of living objects in old generation regions. G1 starts this process in preparation for space reclamation with mixed collection

Some phases of the marking cycle are executed in parallel with the application. During this jvm can continue allocation memory if it needs to. If the collector finishes the marking cycle successfully it will typically move to a space reclamation phase where it runs multiple mixed collections, so called because they evacuate objects from a mix of old and young regions.

When G1 establishes that the mixed collections have evacuated enough old regions without crossing the time limit boundary (maximum wanted duration of the stop-the-world pauses) then it begins the young only phase.

If in other hand, G1 collector doesn't have enough memory to finish the cycle of marking, it might need to run the full garbage collection. Full GC usually takes longer than young only or mixed collections considering that it's evacuation objects across the entire heap, instead of strategically chosen regions.

We can monitor how often is GC run by collection and analyzing GC logs, which I will also cover in future posts.

Stop-The-World pauses are pauses when all application activity stop, and the usually happen when GC is evacuation living objects into different regions and compresses them to free up more memory.

In the next post there I will break down monitoring JVM metrics, but it's important to understand how memory works to understand the statistics provided by monitoring.