Flawed AI Tests at Irregular Triggered Multiple Rogue Breaches

A series of recent cybersecurity incidents involving artificial intelligence agents from OpenAI, Meta, Anthropic, and Google have been traced to a single third-party evaluator: Irregular, an Israeli startup founded in 2023 as Pattern Labs. The revelations have intensified industry-wide concerns regarding AI safety and the reliability of automated testing protocols. According to Irregular co-founder and CTO Omer Nevo, the breaches stem from a shared flaw in one evaluation scenario rather than isolated vulnerabilities within the proprietary models themselves. During the tests, AI agents were deployed in controlled environments designed to simulate realistic cyberattack conditions. The incidents followed a consistent pattern: agents inadvertently gained access to the open internet and targeted real-world systems due to a naming collision between a fictional simulation domain and an active internet address. While the exact extent of external damage remains unconfirmed, the incidents were independently disclosed by the affected tech firms around late July, with OpenAI and Anthropic issuing public statements while Meta and Google became known through subsequent media reports. Irregular also conducted similar evaluations on self-hosted Chinese models from Moonshot AI and Z.ai, including Kimi K3 and GLM-5.2. Nevo confirmed that these tests did not produce comparable breaches, though he cautioned that the absence of incidents does not indicate lower susceptibility to the identified vulnerability. In response to the failures, Irregular has implemented immediate operational changes, including stricter internet access controls, expanded monitoring protocols, and enhanced pre-evaluation verification procedures. The startup has also formalized parameter documentation with clients to prevent scope misalignment. Nevo stated that the evaluation environment issues have been fully resolved and announced plans to publish a comprehensive report detailing lessons learned and standardized safety practices for future cyber assessments. The company emphasized that its disclosures were made to clients and relevant authorities, though public transparency around the timeline and severity remains limited. Major AI developers have not provided detailed responses regarding potential legal recourse or their continued partnership with Irregular. OpenAI and Meta directed inquiries to previously published posts, while Google and Anthropic did not respond. The incidents underscore the growing tension between rapid AI capability testing and rigorous safety validation, highlighting the need for standardized oversight in third-party evaluation workflows. As regulatory frameworks for artificial intelligence evolve, these events are likely to influence how technology firms contract independent security testers, mandate sandbox isolation, and enforce real-time containment protocols to prevent automated agents from escaping controlled environments.