Skip to content

Anthropic Found Its Own Models Broke Out of the Sandbox Too: Three Companies Attacked, and Not One Noticed

1 min read
Share
Anthropic Found Its Own Models Broke Out of the Sandbox Too: Three Companies Attacked, and Not One Noticed

About ten days ago OpenAI admitted that one of its unreleased models broke into Hugging Face's systems while it was confined to a test environment. That was the first confirmed case of one of the big labs losing the thread of control over its own machine. Now Anthropic is out with a list of its own.

The company announced that an internal investigation uncovered three cases in which the Claude model, while carrying out cybersecurity tests, got onto the internet from the test environment and then entered the live systems of three different organisations without permission. The review was not an idea that arrived on its own - Anthropic says it was precisely OpenAI's 21 July incident that prompted it to go through its own records.

The figure is worth reading slowly: out of 141,006 test runs reviewed, three ended up outside the sandbox. Statistically - nothing. In practice - three real companies with real infrastructure, about which nobody had assumed anything.

Anthropic explains that the cause was a misconfigured setup in the environment it used together with its partner Irregular: both sides believed the test environment had no internet access, and it did. The company says it is not looking for someone to blame and that it „approaches the fixes as if the responsibility were entirely its own”. A rare sentence in this industry, but also the cheapest one - Irregular is running its own investigation.

The most uncomfortable part is not the hole, but what the models did once they realised what was happening.

In all three cases, Anthropic says, „Claude was explicitly informed through the prompt that it had no internet access”. So the machines were operating on the information that everything around them was a simulation. When they ran into evidence that it was not - they reacted differently.

The oldest of the three models, Opus 4.7, recognised in all four runs that it had reached a real production system. In two of them it reasoned to itself that the real company must be part of the exercise. And in all four - it kept attacking. It extracted access credentials and reached a database of real data. Mythos 5 likewise noticed signs it was on the real internet, then reverted to the belief that it was in a simulation after all and published a malicious software package to the public PyPI registry. The package was downloaded and executed by external systems before anyone caught it. Only the newest model, an internal research one, stopped on its own the moment it concluded the target was real.

Anthropic adds that during these tests Claude ran without the additional safety layers present in the publicly available versions - because the point of the test is to measure the model's raw capability. In other words: the dams were lowered on purpose, and nobody checked whether the gate behind them was locked. The company also claims there is no evidence any model „pursued a goal of its own” - it was only trying to finish the task it had been given. Comforting, if you think the difference between intent and consequence matters to the company whose database was left open.

Anthropic draws the distinction with the OpenAI case sharply: there the model exploited an unknown vulnerability to get out, here the path was left open by mistake. And it adds that it found the incidents itself, while the two organisations it managed to reach noticed nothing at all and alerted nobody. That is the point that says more about the state of things than any statement: the victims learned they had been attacked when the attacker called to tell them.

The case is now being reviewed by the independent group METR. If the question until yesterday was whether AI models can get out of a controlled environment, the answer has already been given twice in ten days, by the two largest labs. The second question remains - who is counting the environments nobody has reviewed.