What does OOMKilled mean in Kubernetes?

OOMKilled means the Linux kernel's out-of-memory killer terminated the container process because it tried to use more memory than its cgroup allowed. In Kubernetes that cgroup limit is the container's `resources.limits.memory`. The container shows `State: Terminated, Reason: OOMKilled, Exit Code: 137` (128 + signal 9). It is a hard kill, not a graceful shutdown.

What is exit code 137?

Exit code 137 is 128 + 9, meaning the process received SIGKILL. In a container context the usual sender is the kernel OOM killer enforcing the cgroup memory limit, so 137 almost always means OOMKilled. Confirm with `kubectl describe pod` — the Last State reason will say OOMKilled if it was memory rather than a manual kill.

How do I stop a pod being OOMKilled?

First decide whether the limit is too low or the app uses too much memory. Raise `limits.memory` only if the working set is genuinely that large; otherwise fix the real cause — a memory leak, an unbounded cache or query result, or a runtime (JVM, Node.js, Python) whose heap is not sized for the container limit. Setting requests == limits (Guaranteed QoS) prevents eviction under node pressure but does not fix a leak.

Is OOMKilled the same as node memory pressure eviction?

No. OOMKilled is per-container: one container exceeded its own cgroup limit and the kernel killed it. Node MemoryPressure eviction is the kubelet reclaiming memory across the whole node by evicting whole pods (based on QoS and usage-above-requests) when the node itself is running out. They have different signals, different blast radius, and different fixes.

Kubernetes OOMKilled: Root-Cause Troubleshooting for Production Pods

September 7, 2026

OOMKilled with exit code 137 is the kernel telling you a container crossed its memory limit and was killed with SIGKILL. It is abrupt and non-negotiable — no graceful shutdown, no flush, no chance to finish in-flight requests. The instinct is to bump limits.memory and move on, but that only masks the two questions that actually matter: is the limit wrong, or is the app wrong? This guide walks the mechanism and the methodology to answer that correctly instead of playing limit-whack-a-mole.

What actually happens: cgroups and the OOM killer

A container's resources.limits.memory becomes a cgroup memory limit on the host. When the processes in that cgroup try to allocate past the limit and the kernel cannot reclaim enough reclaimable memory (page cache, etc.), the kernel OOM killer fires and kills the largest offending process in that cgroup. Kubernetes reports it as:

    State:      Waiting  (CrashLoopBackOff)
    Last State: Terminated
      Reason:   OOMKilled
      Exit Code: 137

137 = 128 + 9 (SIGKILL). The key implications:

The limit is a hard ceiling. Unlike CPU (which throttles), memory over-limit means death.
It is scoped to the container cgroup, so a container can be OOMKilled while the node has gigabytes free.
Page cache counts toward the cgroup's memory usage, which is why heavy file I/O can appear to inflate "memory" even without a leak.

Step 1: confirm it is OOMKilled, and which container

kubectl describe pod <name> -n <ns>   # read Last State / Reason / Exit Code per container
kubectl get pod <name> -o jsonpath='{.status.containerStatuses[*].lastState}'

In a multi-container pod, identify which container died — the sidecar and the app have separate limits and separate cgroups. Exit code 137 without Reason: OOMKilled can also be an external SIGKILL (e.g. a failed liveness probe escalating, or a manual delete), so check the reason string, not just the code.

Step 2: separate "limit too low" from "app uses too much"

This is the fork in the road. Look at the working set over time, not a single snapshot:

kubectl top pod <name> --containers          # instantaneous
# better: the container_memory_working_set_bytes series in Prometheus/Grafana

Interpret the shape of the curve:

Memory-over-time shape	Diagnosis	Direction
Flat, sits just under limit, killed at peak load	Limit genuinely too low for the working set	Raise limit to peak + headroom
Sawtooth that keeps climbing across requests	Leak or unbounded growth (cache, connections, goroutines)	Fix the app — a bigger limit only delays the kill
Spikes on specific inputs (large upload, big query)	Unbounded per-request allocation	Stream/paginate; cap request size
Steady, but killed right after start	Runtime heap sized above the container limit	Size the runtime to the cgroup (Step 3)

The rule: a monotonic climb is a leak and no limit will save you; a flat line at peak is a sizing problem you fix with the limit.

Step 3: runtimes that ignore the cgroup limit

The most common "mystery" OOMKill is a managed runtime whose default heap is based on the host's total memory, not the container limit:

JVM — modern JVMs are container-aware and read the cgroup limit, but only if you let them. Prefer -XX:MaxRAMPercentage=75.0 over a fixed -Xmx so the heap tracks the limit. Remember the JVM also needs off-heap: metaspace, thread stacks, direct byte buffers, GC structures. A 1Gi limit with -Xmx1g is an OOMKill waiting to happen — the heap alone consumes the whole cgroup.
Node.js — the V8 old-space default (~2GB historically, higher on big hosts) can exceed a small container limit. Set --max-old-space-size to comfortably below the container limit (e.g. ~75%).
Python — no single heap cap; watch for large DataFrames, model weights loaded eagerly, and multiprocessing workers that each carry a full copy of memory.

What I check first for a runtime: does the configured heap + expected off-heap fit inside the container limit with headroom? If heap ≈ limit, that is the bug.

Step 4: requests, limits and QoS

OOMKilled is about limits, but the requests/limits relationship governs eviction behaviour:

Guaranteed (requests == limits) — most protected from node-pressure eviction, but still OOMKilled if it exceeds its own limit.
Burstable (requests < limits) — the pod can use more than it reserved, which is efficient but means memory the scheduler didn't account for; under node pressure these are evicted before Guaranteed.
BestEffort (no requests/limits) — first to be evicted, and the cgroup has no memory ceiling, so it can trigger node-level OOM. Avoid in production.

For latency-critical, memory-stable services, setting requests == limits both stabilises scheduling and removes eviction surprises. It does not fix a leak.

Step 5: OOMKilled vs node MemoryPressure eviction

Do not confuse the two:

OOMKilled — one container exceeded its own cgroup limit; kernel kills the process; Reason: OOMKilled. Fix at the container.
Eviction — the node is low on memory; the kubelet proactively evicts whole pods (BestEffort/Burstable-over-requests first) and the pod shows Status: Evicted with a The node was low on resource: memory event. Fix at the node — right-size requests so the scheduler stops overcommitting, or add capacity.

Common wrong approaches

Reflexively doubling the limit. For a leak this just moves the kill 30 minutes later and hides the regression.
Setting -Xmx equal to the container limit. Leaves nothing for off-heap; the JVM OOMKills before it ever hits its own heap limit.
Removing the limit entirely. Now one leaky pod can take down the node and its neighbours — a much worse incident.
Ignoring the restart count. A pod quietly OOM-restarting every few minutes drops in-flight requests and corrupts latency SLOs long before anyone declares an incident.

Related resources

Kubernetes pod Pending: a production troubleshooting decision tree — the pre-scheduling counterpart to this post-scheduling failure.
Golden Signals dashboard for SRE — how memory kills surface as saturation and error-rate signals.
Kubernetes job support guide, SRE job support guide and the DevOps job support guide.

If a service is OOM-restarting in production right now, real-time proxy job support can put a senior engineer alongside you to read the memory curve and decide leak-vs-sizing quickly. If you are being interviewed on exactly this kind of production reasoning, it maps directly to what a DevOps proxy interview support session prepares you for.

Last reviewed: September 2026.