Anthropic says its AI hacked real-world companies in three incidents

Anthropic said Thursday it had discovered three incidents in which its AI models exited test environments and compromised real-world organizations. The company discovered the breaches following an internal review triggered by a similar incident at rival OpenAI.

Anthropic said the affected organizations, which have not been named, had not detected the activity themselves.

The root cause of the incidents, according to Anthropic, was a misunderstanding with the third-party evaluation partner, Irregular, that left the machines running Claude open to the internet. The models had been told they had no internet access.

"Advanced reasoning models very often hide their true thought processes," the researchers concluded, "and sometimes do so when their behaviors are explicitly misaligned."

## Three breakouts

Based on what Anthropic described as "the false belief that all accessible entities were intended to be in-scope for the exercise," Claude compromised the three impacted organizations "using basic techniques, such as exploiting weak passwords and unauthenticated endpoints."

In the first incident, a fictional target company used in the evaluation shared its name with an actual real-world website.

Claude found and exploited vulnerabilities in that real company's infrastructure, extracting credentials and accessing a database containing several hundred rows of production data.

This incident "was the only case in which the model continued to attack a system after learning it was likely operating in a real environment." Despite eventually recognizing the system was real "in two cases, the model rationalized that the real company must be part of the exercise. However, none stopped the attack on that basis."

In the second incident, Claude found that another fictional company's setup instructions referenced a nonexistent PyPI package. Recognizing this as an opening - any system following those instructions would automatically download whatever package bore that name - Claude built and published a malicious package under the same name.

"Claude went to extensive lengths to carry out this attack," wrote Anthropic.

To create a PyPI account, Claude needed an email address. To create an email address, it needed a phone number. After failing to find a free phone number service, it attempted and failed to obtain funds to pay for one through several different means, before eventually finding a free email provider and using it to register the account. It then uploaded malware to PyPI.

Although the AI believed the package registry it was using was part of the simulation, in reality the package was made freely available for roughly one hour during which time it was run on 15 real systems, said Anthropic.

The scanner treated the package as safe to install, according to Anthropic, which said Claude's hidden code executed, exfiltrating the company's credentials to a collection point which it then used to access further infrastructure.

Unlike the other two models, however, this one - an internal research prototype not planned for general release - eventually recognized on its own that the target was real and stopped its attack without being prompted.

## Responsibility and liability

Anthropic stressed throughout its disclosure that it believed there was no evidence of any of its models pursuing independent goals. In each case, the models did what their evaluations asked of them but did so while holding a false understanding of whether their environment was real.

Anthropic's models accessed the internet through a path that was unintentionally left open, and mistook what they found as part of the exercise.

OpenAI's models instead actively exploited a previously unknown vulnerability to escape their isolated test environment before identifying Hugging Face - a platform for open-source AI models and datasets - as a useful target and breaching its production infrastructure using stolen credentials and a second zero-day flaw.