Postmortem · CVE-2024-0132 · CVSS 9.0

CVE-2024-0132 — NVIDIA Container Toolkit TOCTOU Container Escape Postmortem

A race condition in GPU library mounting let a crafted container image take over the host. Here is exactly what happened and how to close it.

If you run GPUs on Kubernetes, the NVIDIA Container Toolkit mounts host GPU libraries into your Pods — and in affected versions a malicious image could abuse that to escape the container and own the node.

CVE-2024-0132 was a Time-of-Check Time-of-Use (TOCTOU) race condition in how the NVIDIA Container Toolkit mounted host GPU libraries into containers. A maliciously crafted container image could win the race and trick the toolkit into mounting the entire host root filesystem into the container, yielding arbitrary code execution on the host, privilege escalation, and full container escape. Because GPU nodes run privileged, trusted workloads, this was especially dangerous on shared or multi-tenant AI clusters. Wiz Research, which reported it, estimated the toolkit ran in a very large share of cloud GPU environments. The record and fix are below.

Get Instant Help Now Talk to Expert Now

What We Offer

Expert Support for Every IT Challenge

From daily job support to emergency production fixes, proxy interview guidance, and interview coaching — we have the expert for your specific need.

Real-Time Kubernetes AI Job Support

Live expert help during your working hours — running LLM inference (vLLM, KServe, Dynamo), agent runtimes and sandboxes, GPU scheduling, autoscaling, RAG pipelines, and daily platform deliverables on your real cluster so you always hit your deadlines.

Production AI Incident Support

On-call firefighting for live incidents — GPU Pods stuck Pending, CUDA/OOMKilled crashes, vLLM out-of-memory, high TTFT, model-loading failures, autoscaling that will not scale, agent loops, MCP authorization errors, and RAG/vector-DB latency — with an engineer on the call.

Interview & Candidate Marketing

Kubernetes AI proxy interview assistance, profile positioning, and candidate marketing for Platform Engineer, AI Infrastructure Engineer, GPU Infrastructure Engineer, MLOps/LLMOps, and SRE roles — real-time interview guidance, recruiter readiness, and profile visibility.

Real Situations

Incident Record

These are the real-world situations our experts resolve every day — for job support and interview assistance.

DATE: Disclosed September 2024 (reported to NVIDIA 1 Sep 2024; patched mid-September 2024, NVIDIA Security Bulletin 5582).

PLATFORM: Kubernetes and Docker GPU nodes using the NVIDIA Container Toolkit (any cloud or on-prem).

COMPONENT: NVIDIA Container Toolkit ≤ 1.16.1 and NVIDIA GPU Operator ≤ 24.6.1.

WHAT HAPPENED: A TOCTOU race in GPU-library mounting let a malicious container image trick the toolkit into mounting the host root filesystem (/) into the container.

IMPACT: Full container escape — arbitrary code execution on the host, privilege escalation, and node/cluster compromise (CVSS 9.0, Critical).

ROOT CAUSE: Time-of-Check Time-of-Use race condition between validating and mounting host GPU paths, exploitable by a crafted image the toolkit executes.

MITIGATION: If you cannot patch immediately, restrict who can run arbitrary/untrusted container images on GPU nodes, use stronger isolation (gVisor/Kata) for untrusted workloads, and enforce image provenance via admission policy.

FIX: Upgrade NVIDIA Container Toolkit to 1.16.2 or later and NVIDIA GPU Operator to 24.6.2 or later.

OPERATIONAL LESSON: The GPU container runtime is privileged, trusted infrastructure — track its version like a kernel, and never run untrusted images on GPU nodes without strong isolation.

Global Reach

Real-time Kubernetes AI infrastructure support for engineers across USA, Canada, UK, Ireland, Germany, Netherlands, Switzerland, Australia, New Zealand, Singapore, UAE, and worldwide.

Available across US, Canada, UK, European, Australian, and Asia-Pacific business hours — and 24/7 for production incidents.

In-house experts — no sub-contracting or outsourcing

24/7 availability for urgent job support and interview needs

Confidential & professional — NDA available on request

Same-day onboarding for most job support and interview cases

Combined job support + proxy interview service available

Expert Help Available

Need real-time IT job support or interview help? Our experts are available 24/7 — USA, Canada, UK, Europe & worldwide.

Get Instant Help Call Now

FAQ

Frequently Asked Questions

Everything you need to know before getting started with job support or interview assistance.

Ask on WhatsApp

A TOCTOU race in GPU-library mounting let a malicious container image trick the toolkit into mounting the host root filesystem (/) into the container. Full container escape — arbitrary code execution on the host, privilege escalation, and node/cluster compromise (CVSS 9.0, Critical). You are likely affected if you run NVIDIA Container Toolkit ≤ 1.16.1 and NVIDIA GPU Operator ≤ 24.6.1. at the versions noted in the record below. We can audit your cluster against this and the wider class of AI-infrastructure risks and tell you precisely where you are exposed.

Fix: Upgrade NVIDIA Container Toolkit to 1.16.2 or later and NVIDIA GPU Operator to 24.6.2 or later. Mitigation if you cannot patch immediately: If you cannot patch immediately, restrict who can run arbitrary/untrusted container images on GPU nodes, use stronger isolation (gVisor/Kata) for untrusted workloads, and enforce image provenance via admission policy. We help you apply the fix safely in production — staged rollout, verification, and the admission/network guardrails that reduce blast radius for the next issue of this class.

The GPU container runtime is privileged, trusted infrastructure — track its version like a kernel, and never run untrusted images on GPU nodes without strong isolation. This is why we treat the AI-infrastructure supply chain, container runtime, and admission path as security-critical — not just the application layer.

Yes. We run a focused review of your container runtime (NVIDIA Container Toolkit / GPU Operator versions), ingress and admission webhooks, model and image supply chain, agent/tool sandboxing, and RBAC/network policy — mapping each finding to a concrete fix and a guardrail. See our Kubernetes AI security hub.

Both. This page documents a real, publicly disclosed incident with its official source so you can act on it. If you would rather an engineer work it with you — patching safely in production, or auditing for the wider class of risk — that service is available same-day and confidentially.

Official Source

Wiz Research disclosure and deep-dive on CVE-2024-0132, and NVIDIA Security Bulletin 5582. Verify affected and fixed versions against the vendor advisory before patching.

Read the Wiz Research disclosure (CVE-2024-0132)

Get Started Today

Exposed to CVE-2024-0132 or Want a Cluster Security Review?

In-house Kubernetes, GPU, and AI-infrastructure security engineers available same-day — safe production patching, blast-radius review, and hardening against this class of risk. Talk to ProxyTechSupport on WhatsApp now.

Get Instant Help Now Talk to Expert Now

+91 96606 14469

support@proxytechsupport.com

Proxy Tech Support provides interview preparation, technical guidance, and job support services. All services are advisory and educational in nature.