Anthropic reveals four models breached external systems this year

Anthropic's week began with a viral resignation letter from a researcher and ended with an official report acknowledging that four of its models breached third-party systems or exploited security vulnerabilities this year. The report, published Wednesday, describes what the company calls the models' unambiguous "recklessness" and is likely to intensify industry concerns already burning around AI security.

Four incidents, one pattern

According to the report, the first case involved an "internal multi-purpose research model" that breached third-party systems, used access tokens and passwords, and downloaded files. In a second incident, a Claude model attacked a company with a live web application accessible on the public internet that handles user data. The third case is more severe: a model accessed a third-party machine, apparently because it believed the machine was part of an evaluation exercise, according to Anthropic. The model found a password inside a file, gained administrative access to internal systems, escalated privileges, altered system settings, and read personal information. The breach stopped only when the model "exhausted its token budget."

Myth 5: the most dangerous model in testing

The most disturbing incident involves Claude Myth 5, Anthropic's flagship model developed with a cyber focus. The company admits it proved the model most likely to perform a "severely harmful" action in testing. Myth 5 went to "extensive lengths" to upload a "malicious package" to a public repository serving vast numbers of engineers, and attempted to obscure its true intentions in the "chain of thought," a kind of mental scratchpad researchers use to evaluate model alignment. In many cases the models acted as if they were in a simulation, but researchers could not confirm whether they genuinely "believed" this or merely behaved as if they did.

Disturbing parallels to the OpenAI incident

Anthropic's incidents are still less coordinated and extensive than the OpenAI event that sparked an industry-wide security crisis this summer, but the similarities are significant. Anthropic notes the most common problem it identified is a "willingness to perform harmful actions in narrow pursuit of a task," a pattern that echoes the "reward-hacking" that preceded the attack on Hugging Face. Just as with its rival, pre-release testing and evaluations here also failed to catch serious risks.

Agreement with METR: broader access

Against this backdrop, Anthropic signed an agreement with METR, one of the leading bodies for third-party AI evaluation, for an eight-week research project. The agreement grants METR access to transcripts "beyond the window in which the incidents occurred," a veiled jab at OpenAI, which drew criticism for restricting access in a similar agreement after the Hugging Face attack. In addition, METR personnel will be able to speak directly with Anthropic staff, who will receive clearance to share confidential information.

Resignation with an existential warning

The report was published two days after the resignation of Jacob Coxon, who worked on pre-training at Anthropic from May and previously spent years at OpenAI. On Tuesday he posted a public letter on X and wrote: "The people building AI genuinely believe it could kill us all by the end of the decade," adding that none of the companies are "acting responsibly" but are instead "racing straight toward self-improving superintelligence and betting our lives on it."