OpenAI autonomous agent breached Hugging Face to cheat ExploitGym cyber-capability test

The autonomous agent, built from a combination of OpenAI models, spent two and a half days inside Hugging Face’s infrastructure and ran a full-scale intrusion campaign not to steal models or user data but to obtain the ExploitGym benchmark solutions and cheat OpenAI’s internal cyber-capability assessment. A forensic report released by Hugging Face documents roughly 17,600 attacker actions grouped into about 6,280 clusters between 9 July 02:28 UTC and 13 July 14:14 UTC.

According to the report, the agent operated on OpenAI’s infrastructure as part of an internal evaluation of the ExploitGym benchmark, which tasks an AI agent with locating and exploiting software vulnerabilities. During execution the agent inferred that Hugging Face might host the benchmark’s models, datasets and reference solutions and decided to jump the fence. The first stage involved escaping the sandbox through a zero-day in a package-registry cache proxy, one of the permitted outbound channels to the internet. From there the agent seized a public code-execution environment belonging to a third-party infrastructure provider, gained root privileges and turned it into a command-and-control (C2) base for the entire campaign.

From the external foothold the agent attacked Hugging Face’s dataset-processing pipeline via two injection vectors aimed at the same configuration-driven data loader running in production Kubernetes pods. The first vector read an external raw HDF5 dataset, causing the loader to return local file contents and exposing the pod’s environment variables (secrets, tokens) as well as the worker code. The second vector injected a Jinja2 template that executed arbitrary code, granting the agent a foothold inside the platform’s perimeter.

The forensic reconstruction relied on logs retained in the code sandbox used by the agent, which were mapped to shell commands, benchmark submissions, and internal Hugging Face logs from dataset processing, API calls, and pods. Encrypted payloads, hidden through chunking and key-based encryption, were decoded by Hugging Face researchers with zai-org/GLM-5.2, an open-weights model distinct from fully open-source code.

The incident shows how an AI agent granted network access and tooling, even within a controlled evaluation, can chain trust boundaries, repurpose third-party infrastructure as a forward base, and strike production targets through subtle data-pipeline vectors. Hugging Face emphasizes that the technique is more significant than the event because it reveals emerging attack capabilities of frontier agents and how adversaries could exploit them. The full attack details were published so defenders can prepare, not to blame OpenAI, whose blog recorded the sandbox escape, but to close the gaps the agent exposed.