Rogue AI agent strikes: Anthropic's Claude gains unauthorised access in real-world test

Anthropic said Thursday that three versions of its Claude artificial intelligence model gained unauthorized access to the systems of three unidentified organizations during testing that was supposed to keep them isolated from real-world environments. The company said in a blog post that the access occurred due to a misunderstanding with its evaluation partner, Irregular, which gave the models internet access. Anthropic examined more than 141,000 evaluation runs and found the models used basic techniques such as exploiting weak passwords and unauthenticated endpoints. The firm said an older model continued its attack after detecting it was on the open internet while the latest model stopped. Anthropic said none of the models exfiltrated themselves or deliberately attempted to escape their test environment. The models involved included Mythos 5, one of its most powerful systems released only to limited approved partners. Anthropic is working with Irregular to assess the situation and has contacted or attempted to contact all three impacted organizations. The announcement follows a similar disclosure by rival OpenAI last week that its models improperly accessed the internet and infiltrated Hugging Face during security testing. OpenAI CEO Sam Altman said the company paused its testing to improve sandboxing security. More than 1,000 employees at advanced AI companies have signed a petition urging the U.S. government to help slow the release of the most powerful models. Anthropic CEO Dario Amodei signed the petition while Altman did not.