A pod in Pending is one of the most common — and most misdiagnosed — states in Kubernetes. Pending means the API server has accepted the pod but it is not yet bound to a node and running. That is a narrow window: it rules out image pulls, container crashes and readiness probes (those happen after scheduling) and points you at one of four families of cause — the scheduler cannot fit the pod, a placement constraint excludes every node, storage has not bound, or the cluster is out of capacity and waiting on the autoscaler. This guide is the decision tree a reliability engineer actually walks, in order.
Step 0: read the events, not the status
The pod phase (Pending) tells you almost nothing. The events tell you everything. Start here every time:
kubectl describe pod <name> -n <namespace>
# scroll to the Events section at the bottom
kubectl get events -n <namespace> --sort-by=.lastTimestamp
A pod that cannot be scheduled emits a FailedScheduling event from the default-scheduler with a per-node tally, for example:
0/6 nodes are available: 3 Insufficient cpu, 2 node(s) had untolerated taint
{node-role.kubernetes.io/control-plane: }, 1 node(s) didn't match
Pod's node affinity/selector.
That single line usually ends the investigation — it tells you exactly how many nodes failed for each reason. If there is no FailedScheduling event at all, the pod is not stuck on the scheduler; jump to the storage branch (Step 4) or check whether the scheduler itself is healthy (Step 6).
Step 1: resource requests vs allocatable — "Insufficient cpu/memory"
This is the number-one cause. The scheduler places pods by their requests, not their limits and not their actual usage. A node with 60% idle CPU can still reject a pod if its requested CPU is already committed to other pods.
- Check what the pod asks for:
kubectl get pod <name> -o jsonpath='{.spec.containers[*].resources.requests}'. - Check what nodes can offer:
kubectl describe node <node>and readAllocatableand theAllocated resourcestable (requests vs limits already committed). - Common trap: a request of
cpu: 4on a cluster whose largest node is 4 vCPU — after system daemons and kube-reserved, no single node has 4 whole cores allocatable, so the pod is unschedulable forever, not just temporarily.
What I check first: is the request physically satisfiable by any single node? If not, this is a manifest bug (right-size the request) or a node-pool sizing problem — no amount of waiting fixes it.
Step 2: node selectors, affinity and topology spread
If the event says didn't match Pod's node affinity/selector or didn't match pod topology spread constraints, a placement rule is excluding nodes:
nodeSelector/nodeAffinity— the pod demands a label (e.g.disktype=ssd,gpu=true, a specifictopology.kubernetes.io/zone) that no available node carries. Verify withkubectl get nodes --show-labels.podAntiAffinitywithrequiredDuringScheduling— a hard "do not co-locate" rule that leaves no legal node once replicas are spread. This bites when you scale replicas past the number of eligible nodes/zones.topologySpreadConstraintswithwhenUnsatisfiable: DoNotSchedule— the pod cannot be placed without violating the spread budget across zones.
The fix is almost always to relax a required rule to preferred, or to add nodes/labels in the zone the constraint demands.
Step 3: taints and tolerations — "untolerated taint"
Nodes advertise taints; pods must carry a matching toleration to land on them. If every fitting node is tainted (GPU pools, control-plane nodes, spot pools tainted for eviction), the pod stays Pending.
kubectl describe node <node> | grep -i taint
# add a toleration to the pod spec, or remove/adjust the taint on the node
Watch for the built-in taints Kubernetes adds during trouble: node.kubernetes.io/not-ready, node.kubernetes.io/unreachable, node.kubernetes.io/disk-pressure, memory-pressure. If nodes are tainted with a pressure condition, the real problem is node health, not your toleration.
Step 4: storage — PersistentVolumeClaim not bound
A pod that mounts a PVC will not schedule until that claim is Bound. Check the claim, not just the pod:
kubectl get pvc -n <namespace>
kubectl describe pvc <claim> -n <namespace>
Typical failures:
- No matching StorageClass / no default — dynamic provisioning never fires; the PVC sits
Pendingwith aProvisioningFailedor "no persistent volumes available" event. volumeBindingMode: WaitForFirstConsumer— this is expected: the volume binds only once a pod is scheduled, so PVC-Pending and pod-Pending can be a chicken-and-egg that resolves as soon as a node is chosen. If it never resolves, it usually collapses back to a Step 1/2 scheduling problem in the target zone.- Zone mismatch — an EBS/PD volume already provisioned in
us-east-1acannot attach to a node in1b; the scheduler correctly refuses. This is a very common multi-AZ trap.
Step 5: capacity and the cluster autoscaler
If the pod fits logically but there is simply no room, a healthy cluster autoscaler (or Karpenter) should add a node within a minute or two. Confirm it is actually trying:
kubectl get events --sort-by=.lastTimestamp | grep -i -E "scale|autoscaler|provision"
# Karpenter:
kubectl logs -n karpenter deploy/karpenter | tail -50
Reasons a scale-up never happens: the node group is at its max size; the cloud provider is out of the requested instance type or hit a quota; the pod requests a resource no node shape in the group can provide (e.g. a GPU the group has none of); or the autoscaler is scoped away from the relevant node group. Karpenter will log a specific reason such as incompatible with nodepool or an instance-type constraint — read it verbatim.
Step 6: the scheduler itself, quotas and admission
If there is no FailedScheduling event and no scale activity, look one layer up:
- ResourceQuota — a namespace quota that is exhausted blocks admission with a clear message;
kubectl describe quota -n <ns>. - Missing PriorityClass / preemption — low-priority pods can be stuck behind higher-priority workloads that own the capacity.
- Scheduler unhealthy — on self-managed clusters, confirm
kube-scheduleris running (kubectl get pods -n kube-system). No scheduler means nothing gets bound and every new pod is Pending.
The decision tree, condensed
| Event / symptom | Most likely cause | First action |
|---|---|---|
Insufficient cpu/memory | Requests exceed allocatable | Right-size requests or add/upsize nodes |
didn't match node affinity/selector | Label/affinity rule too strict | Fix label or relax required→preferred |
untolerated taint | Node tainted, pod lacks toleration | Add toleration or clear taint; check pressure taints |
topology spread violation | Spread budget can't be met | Add nodes in the deficit zone or relax spread |
PVC Pending | No StorageClass / zone mismatch / WaitForFirstConsumer | Fix StorageClass, align zones |
No FailedScheduling, no scale-up | Autoscaler capped / wrong instance type / scheduler down | Check autoscaler logs, quotas, kube-scheduler health |
Common wrong approaches
- Deleting and recreating the pod. A Deployment reschedules the identical spec into the identical constraints — you get the same Pending pod with a new name.
- Raising
limits. Limits do not affect scheduling; onlyrequestsdo. Raising limits changes nothing about a Pending pod. - Blaming the image or the registry. Those show as
ImagePullBackOffafter scheduling — a different problem entirely. - Waiting. If the request is physically unsatisfiable, waiting is infinite. Decide early whether the cause is transient (capacity) or structural (manifest/node sizing).
Related resources
- Kubernetes OOMKilled: root-cause troubleshooting — the other end of the lifecycle, once the pod is finally running.
- CI/CD pipeline failures: application vs infrastructure vs runner — when the Pending pod is actually your build agent.
- Kubernetes job support guide and the DevOps job support guide for the wider operational context.
If you are debugging a Pending pod live in a production cluster and the clock is running, real-time proxy job support puts a senior Kubernetes engineer on the screen share with you. And if you are being asked to reason through exactly this decision tree in a technical screen, that is the core of a DevOps proxy interview support session.
Last reviewed: September 2026.