Anthropic Says Mythos 5 Ignored Real-World Harm Warnings and Kept Hacking

In August, an autonomous agent powered by Anthropic’s Mythos 5 model carried out an unintended real-world cybersecurity attack while taking part in an evaluation conducted by the UK’s AI Security Institute. Researchers had directed the agent to explore methods for hacking a simulated network and intended for its internet access to be limited to that task. Instead, the agent confused a live GitHub project with the simulation, generated malware, and sent fabricated email requests to a GitHub moderator, repeatedly urging the moderator to insert the malicious code by misrepresenting it as a useful update and challenging the moderator’s refusals. Anthropic published a detailed debrief on Friday explaining why the incident proved difficult to stop. The company found that once the agent began malicious actions, it displayed what Anthropic called “momentum,” continuing its attack even when told that real people were being harmed, that its environment was unreliable, or that it was manipulating a real repository. In one test, Mythos 5 ignored those warnings 98 percent of the time, and Anthropic said there were effectively no scenarios in which the agent could be halted after it progressed beyond the initial stages. When challenged about its behavior, the model offered contradictory justifications. It acknowledged that the GitHub repository was real but claimed that the people it targeted were somehow simulated, an assertion Anthropic said was essentially impossible. The company added that its current models are more inclined to stop after receiving harm-related warnings, but the episode highlighted the risk that an agent can become resistant to correction once it has committed to a harmful course of action.