Postmortem · Model Supply Chain · Ongoing

Hugging Face Malicious-Model Supply-Chain Postmortem — Pickle RCE & the nullifAI Pattern

Downloading a model can run code. Pickle-based model files execute arbitrary Python on load — and attackers have shipped backdoored models to public hubs. Here is the record and the defenses.

Private AI on Kubernetes almost always pulls open-weight models from public hubs. If those models are pickle-serialized, loading one can execute arbitrary code inside your inference Pod — and malicious models have repeatedly reached the Hugging Face Hub.

This is a recurring supply-chain class rather than a single CVE. PyTorch’s default save format wraps weights in Python’s pickle module, and pickle executes arbitrary Python on load: an attacker overrides __reduce__ on a custom class so that torch.load() runs a payload as a side effect of "deserializing" the model. Researchers have repeatedly found backdoored models on the Hugging Face Hub — ReversingLabs’ "nullifAI" campaign (February 2025) shipped reverse shells in pickle models and evaded the platform’s picklescan by using 7z archives and intentionally malformed pickles that picklescan failed to parse but Python still executed; JFrog flagged 100+ malicious models in early 2025; and earlier 2024 rounds found the same pattern. Hugging Face removed flagged models quickly, but the lesson stands: a model file is executable code. The record and defenses are below.

Get Instant Help Now Talk to Expert Now

What We Offer

Expert Support for Every IT Challenge

From daily job support to emergency production fixes, proxy interview guidance, and interview coaching — we have the expert for your specific need.

Real-Time Kubernetes AI Job Support

Live expert help during your working hours — running LLM inference (vLLM, KServe, Dynamo), agent runtimes and sandboxes, GPU scheduling, autoscaling, RAG pipelines, and daily platform deliverables on your real cluster so you always hit your deadlines.

Production AI Incident Support

On-call firefighting for live incidents — GPU Pods stuck Pending, CUDA/OOMKilled crashes, vLLM out-of-memory, high TTFT, model-loading failures, autoscaling that will not scale, agent loops, MCP authorization errors, and RAG/vector-DB latency — with an engineer on the call.

Interview & Candidate Marketing

Kubernetes AI proxy interview assistance, profile positioning, and candidate marketing for Platform Engineer, AI Infrastructure Engineer, GPU Infrastructure Engineer, MLOps/LLMOps, and SRE roles — real-time interview guidance, recruiter readiness, and profile visibility.

Real Situations

Incident Record

These are the real-world situations our experts resolve every day — for job support and interview assistance.

DATE: Recurring since 2024; ReversingLabs "nullifAI" disclosure February 2025; JFrog 100+ malicious models early 2025.

PLATFORM: Any AI workload (on Kubernetes or elsewhere) that loads pickle-serialized models pulled from public hubs such as Hugging Face.

COMPONENT: Python pickle / PyTorch default serialization (torch.load); models distributed via public model hubs.

WHAT HAPPENED: Attackers published models whose pickle payload executes arbitrary code on load (via __reduce__), using evasion (7z wrapping, malformed pickles) to bypass automated scanners.

IMPACT: Arbitrary code execution inside the inference/training process on model load — reverse shells, credential and data theft, and a foothold in the AI pipeline.

ROOT CAUSE: Pickle deserialization is code execution by design; model provenance and scanning were insufficient, and scanners could be evaded.

MITIGATION: Prefer safetensors over pickle; never load untrusted pickle models; pin and verify model provenance (hashes, signatures); scan models in a sandbox; and isolate model-loading in a least-privilege, egress-restricted Pod.

FIX: There is no single patch — the defense is architectural: safetensors-only policies where possible, provenance pinning, pre-load scanning, and sandboxed/isolated loading of any untrusted model.

OPERATIONAL LESSON: A model file is executable code. Treat the model supply chain like any other dependency supply chain: trusted sources, provenance, scanning, and least-privilege, isolated execution.

Global Reach

Real-time Kubernetes AI infrastructure support for engineers across USA, Canada, UK, Ireland, Germany, Netherlands, Switzerland, Australia, New Zealand, Singapore, UAE, and worldwide.

Available across US, Canada, UK, European, Australian, and Asia-Pacific business hours — and 24/7 for production incidents.

In-house experts — no sub-contracting or outsourcing

24/7 availability for urgent job support and interview needs

Confidential & professional — NDA available on request

Same-day onboarding for most job support and interview cases

Combined job support + proxy interview service available

Expert Help Available

Need real-time IT job support or interview help? Our experts are available 24/7 — USA, Canada, UK, Europe & worldwide.

Get Instant Help Call Now

FAQ

Frequently Asked Questions

Everything you need to know before getting started with job support or interview assistance.

Ask on WhatsApp

Attackers published models whose pickle payload executes arbitrary code on load (via __reduce__), using evasion (7z wrapping, malformed pickles) to bypass automated scanners. Arbitrary code execution inside the inference/training process on model load — reverse shells, credential and data theft, and a foothold in the AI pipeline. You are likely affected if you run Python pickle / PyTorch default serialization (torch.load); models distributed via public model hubs. at the versions noted in the record below. We can audit your cluster against this and the wider class of AI-infrastructure risks and tell you precisely where you are exposed.

Fix: There is no single patch — the defense is architectural: safetensors-only policies where possible, provenance pinning, pre-load scanning, and sandboxed/isolated loading of any untrusted model. Mitigation if you cannot patch immediately: Prefer safetensors over pickle; never load untrusted pickle models; pin and verify model provenance (hashes, signatures); scan models in a sandbox; and isolate model-loading in a least-privilege, egress-restricted Pod. We help you apply the fix safely in production — staged rollout, verification, and the admission/network guardrails that reduce blast radius for the next issue of this class.

A model file is executable code. Treat the model supply chain like any other dependency supply chain: trusted sources, provenance, scanning, and least-privilege, isolated execution. This is why we treat the AI-infrastructure supply chain, container runtime, and admission path as security-critical — not just the application layer.

Yes. We run a focused review of your container runtime (NVIDIA Container Toolkit / GPU Operator versions), ingress and admission webhooks, model and image supply chain, agent/tool sandboxing, and RBAC/network policy — mapping each finding to a concrete fix and a guardrail. See our Kubernetes AI security hub.

Both. This page documents a real, publicly disclosed incident with its official source so you can act on it. If you would rather an engineer work it with you — patching safely in production, or auditing for the wider class of risk — that service is available same-day and confidentially.

Official Sources

ReversingLabs’ "nullifAI" research on malicious ML models on Hugging Face, JFrog’s findings on malicious models, and Hugging Face’s own guidance on pickle risks and safetensors. Verify current platform protections before relying on them.

Read the ReversingLabs malicious-model research (nullifAI)

Get Started Today

Exposed to the Hugging Face malicious-model (pickle) attacks or Want a Cluster Security Review?

In-house Kubernetes, GPU, and AI-infrastructure security engineers available same-day — safe production patching, blast-radius review, and hardening against this class of risk. Talk to ProxyTechSupport on WhatsApp now.