Lesson 20.1 · JVM Internals · 15 min read

The heap is not the footprint

Q: Why does a Java process use more memory than its -Xmx?

Because -Xmx limits only the Java heap, and the JVM is a native program with other allocations that the kernel, the cgroup and the OOM killer all count: - Metaspace - class metadata, not limited by -Xmx (a Spring app with 16 000 classes needs ~100 MB). - Thread stacks - one per thread (NLWP in ps). - Code cache - the JIT's compiled code. - GC structures, symbols, and direct buffers (which default to about the heap size). On oncall-lab orders runs with -Xmx512m and ps -o pid,rss,vsz,nlwp shows ~597 MB RSS, while jcmd 1210 GC.heapinfo shows 362 MB committed heap. 100-400 MB of non-heap is normal; a leak is RSS or after-GC heap that keeps climbing. For operations: the container limit must hold the whole process - rule of thumb limit = heap x 1.3 to 1.5, then measure. VSZ (3.4 GB here) is reserved address space and costs nothing.

In plain words

Imagine a backpack you take on a school trip. Your teacher says "your lunchbox can be at most this big". But the backpack also holds a water bottle, a jumper, a pencil case and a map. If the bus only has room for backpacks of a certain size, the lunchbox rule is not the one that matters: the whole backpack has to fit.

The lunchbox is the Java heap, the part -Xmx limits. The rest of the backpack is metaspace, thread stacks, the code cache and GC data. The bus seat is the cgroup memory limit, and the OOM killer only measures the whole backpack. On oncall-lab, ps shows orders at about 597 MB resident for a 512 MB heap, and jcmd 1210 GC.heap_info shows the heap part from inside.

One process, many memory areas

The problem. A Java service in a pod is killed with exit 137 while every "heap" number the developers show you looks fine. The heap is only part of a Java process's memory, and the kernel counts all of it. To argue with evidence you need to know what else is in there.

What you need to know already: processes, RSS and VSZ (3.1, 5.1), the OOM killer and cgroup limits (5.7, 5.11), the JVM-in-a-container trap (5.13), threads as NLWP (3.1), requests/limits and OOMKilled in Kubernetes (17.1, 17.6).

Java in five sentences (you do not need to write any):

A Java program is written as classes (blueprints: the code and the fields of a kind of thing); at run time the program creates objects (instances of a class: an order, a request, a string) in memory.
The compiler turns the source into bytecode - instructions for an imaginary machine - packed with its libraries into a .jar file (a zip of classes).
The JVM (Java Virtual Machine, the java program) runs that bytecode: first by interpreting it, then the JIT (just-in-time compiler) compiles the busy parts into real machine code while the program runs.
Objects live in the heap, one big memory area the JVM manages. The program never frees objects; a garbage collector (GC) finds objects nothing uses any more (garbage) and reclaims their memory, sometimes pausing the program to do so.
Each thread (one line of execution inside the process - a request being served, a background job) has its own stack for its local variables; the NLWP column of ps counts them.

The JDK (Java Development Kit) is the JVM plus the developer and diagnostic tools (jcmd, jstat...); the JRE is just the runtime. HotSpot is the name of the standard JVM, and OpenJDK the open-source build of it you install with apt.

A JVM is a C++ program that happens to run your bytecode. The Java heap - the thing -Xmx limits - is one of its allocations. The kernel, ps and the cgroup see the whole process, and so does the OOM killer.

the process (RSS)
├── Java heap                  -Xmx / MaxRAMPercentage       your objects
│   ├── young generation       eden + 2 survivor spaces       new objects
│   └── old generation         objects that survived          long-lived data
├── Metaspace                  unbounded by default           class metadata
├── Compressed class space     1 GB reserved, little used     class pointers
├── Thread stacks              ThreadStackSize per thread     one per thread
├── Code cache                 240 MB reserved                JIT-compiled code
├── GC data structures         G1: a few % of the heap        remembered sets, card tables
├── Direct / off-heap buffers  MaxDirectMemorySize (= -Xmx!)  NIO, Netty, gRPC
├── Symbols, strings table     a few tens of MB               interned names
└── malloc arenas, libs        glibc, JNI, libzip             outside the JVM's own count

On this box: /opt/app/orders.jar runs with -Xmx512m. Look at what the kernel says it uses:

$ ps -o pid,rss,vsz,nlwp,cmd -p $(pgrep -f orders.jar)
    PID    RSS     VSZ NLWP CMD
   1210 610962 3404344   56 /usr/bin/java -Xmx512m -jar /opt/app/orders.jar

RSS is in KB: 597 MB resident for a 512 MB heap that is not even full. VSZ (3.4 GB) is address space the JVM reserved - mostly the 1 GB compressed class space and the code cache reservation - and costs nothing until touched. NLWP is the number of threads: 56.

The heap itself, from inside the JVM:

# jcmd ships with the JDK: installed in lesson 3 of this chapter
sudo jcmd 1210 GC.heap_info
1210:
 garbage-first heap   total 370688K, used 252111K [0x00000000e0000000, 0x0000000100000000)
  region size 1024K, 47 young (48128K), 3 survivors (3072K)
 Metaspace       used 98304K, committed 99602K, reserved 1213714K
  class space    used 13312K, committed 13568K, reserved 1048576K

Read it line by line:

garbage-first heap - the collector is G1.
total 370688K - committed heap: 362 MB the JVM has asked the OS for. It grows up to -Xmx as needed and G1 rarely gives it back.
used 252111K - bytes in live objects plus garbage not yet collected. Used is not "live": a heap that is 90% used just before a young GC is normal.
the two hex numbers are the reserved address range: 0xe0000000 to 0x100000000 is exactly 512 MB - that is -Xmx.
region size 1024K - G1 splits the heap into equal regions (1-32 MB, chosen from the heap size). 47 young regions are eden plus survivors.
Metaspace used 98304K - 96 MB of class metadata. Not in the heap, not limited by -Xmx, and a Spring Boot app with 16 000 classes needs it.

So the arithmetic for this process is roughly:

Java heap committed          362 MB
Metaspace + class space      110 MB
Thread stacks (56 threads)    17 MB   (touched pages, not the 2 MB reserved each)
Code cache                    45 MB
GC structures                 22 MB
Symbols, internal, other      35 MB
                            -------
                             ~590 MB  = the RSS above

The number that surprises people is how much is not heap: ~230 MB here. A bigger Spring app with more threads, Netty buffers and 25 000 classes easily carries 400 MB of non-heap.

Heap regions and object lifetime

The generational hypothesis: most objects die young. A request allocates strings, DTOs and buffers, and nearly all of them are garbage by the time the response is written. So the heap is split:

young generation
  eden           every new object is born here
  survivor 0/1   objects that survived one or more young GCs
old generation   objects that survived enough young GCs ("tenuring threshold")
                 or were too big for young (G1 "humongous": > half a region)

A young GC (minor GC) copies the few live objects out of eden into a survivor space and throws eden away wholesale. It costs time proportional to what survives, not to what was allocated - which is why it is cheap and frequent. Objects that keep surviving are promoted to old.

The old generation is collected less often and at higher cost: G1 marks it concurrently and cleans it with mixed collections; if it cannot keep up it falls back to a Full GC that stops everything. The next lessons read those events in logs.

Why this matters for operations

The container limit must hold the whole process, not the heap. -Xmx1g with MemoryMax=1G (chapter 5) or limits.memory: 1Gi is an OOM kill waiting for the first busy hour.
Rule of thumb: limit = heap x 1.3 to 1.5, then measure (NMT, later in this chapter) and adjust.
Direct buffers default to the heap size. MaxDirectMemorySize defaults to roughly -Xmx, so a Netty-heavy service can take another heap's worth outside the heap before the JVM complains.
Threads cost memory outside the heap. A thread leak kills a pod by RSS while the heap graph stays flat - one of this chapter's incidents.

Where each number lives

ps -o rss,vsz,nlwp -p PID                      the kernel's view of the process
cat /proc/PID/status | grep -E 'VmRSS|Threads'  same, from /proc
cat /sys/fs/cgroup/system.slice/UNIT/memory.current   what the cgroup charges
jcmd PID GC.heap_info                          the heap, from inside
jcmd PID VM.native_memory summary              every category (needs NMT on)

The cgroup number (memory.current) is usually a bit higher than RSS: it also counts page cache the process caused, such as log files it wrote. The OOM killer acts on the cgroup number.

Common misreadings

"Used is 90%, we are about to OOM." No - look at used after a GC.
"RSS is bigger than -Xmx, so there is a leak." No - that is normal, by 100-400 MB. A leak is RSS or after-GC heap that keeps climbing.
"VSZ is 3.4 GB, the process is huge." Reserved address space, not memory.
"Setting -Xms = -Xmx makes it use less." It makes the heap fully committed from the start - more RSS early, fewer resizes later. A choice, not a fix.

Why it helps

The most common JVM incident on Kubernetes is a pod set to -Xmx1g with limits.memory: 1Gi. It runs fine in the quiet morning and gets OOMKilled at the first busy hour, and the heap graph never went above 70%. If you know the process is heap plus 100-400 MB of non-heap, you spot this in the PR review before it ships.

It also stops false alarms. A teammate sees VSZ at 3.4 GB and thinks the service is huge, or sees heap used at 90% and wants to page someone. You know VSZ is reserved address space, and "used" includes garbage waiting for the next young GC. You look at RSS against the limit and the heap after GC instead, which is what decides whether something is actually wrong.

Commands in this lesson

ps

FAQ

What is the difference between RSS, VSZ and committed heap?

RSS is the physical memory the process actually has resident; it is what the cgroup charges and what the OOM killer acts on. VSZ is reserved address space, which includes the 1 GB compressed class space and the code cache reservation, and costs nothing until touched. Committed heap is the part of the heap the JVM has asked the OS for, which grows up to -Xmx and rarely shrinks with G1. On orders: RSS about 597 MB, VSZ 3.4 GB, committed heap 362 MB.

Heap used is at 90%. Are we about to run out of memory?

Probably not. Used includes objects that are already garbage but not yet collected. Eden fills up between young GCs, so the heap graph is a sawtooth that often reaches high values right before a collection. The number that matters is heap used right after a GC, the troughs of the sawtooth. If those troughs are stable, the service is fine. If they keep climbing over hours, you are looking at a leak.

What is metaspace and why is it not in the heap?

Metaspace holds class metadata: the structure of every loaded class, its methods and constant pool. It lives in native memory, outside the heap, and is not limited by -Xmx. By default it is unbounded. A Spring Boot app loads 15 000 or more classes, so 100 MB of metaspace is normal. It becomes a problem only when it keeps growing, typically a classloader leak or a library generating classes at runtime.

What are young and old generations for?

They exist because most objects die young. A request creates strings, DTOs and buffers that are garbage by the time the response is written. New objects go to eden; a young GC copies the few survivors out and throws eden away, so its cost depends on what survives, not on what was allocated. Objects that survive several young GCs are promoted to the old generation, which is collected less often and at higher cost.

Does -Xms equal to -Xmx save memory?

No, the opposite. It commits the full heap from the start, so RSS is higher from the first minute. What you gain is no heap resizing later and a predictable footprint, which some teams prefer for latency and capacity planning. It is a trade-off, not a fix for anything. For containers, the percentage flags are usually a better way to express the same intent.

In an interview Mid

Why does a Java process use more memory than its -Xmx?

Because -Xmx limits only the Java heap, and the JVM is a native program with other allocations that the kernel, the cgroup and the OOM killer all count:

Metaspace - class metadata, not limited by -Xmx (a Spring app with 16 000 classes needs ~100 MB).
Thread stacks - one per thread (NLWP in ps).
Code cache - the JIT's compiled code.
GC structures, symbols, and direct buffers (which default to about the heap size).

On oncall-lab orders runs with -Xmx512m and ps -o pid,rss,vsz,nlwp shows ~597 MB RSS, while jcmd 1210 GC.heap_info shows 362 MB committed heap. 100-400 MB of non-heap is normal; a leak is RSS or after-GC heap that keeps climbing.

For operations: the container limit must hold the whole process - rule of thumb limit = heap x 1.3 to 1.5, then measure. VSZ (3.4 GB here) is reserved address space and costs nothing.

Also asked: What is the difference between RSS and VSZ for a Java process? · What are the young and old generations, and why does the heap have them? · Heap usage is at 90%. Should you be worried?

Practise this lesson in the terminal Free, in your browser - a real Ubuntu terminal to try it in, with missions that check your work.