Anthropic’s latest AI model, Claude Sonnet 4.5, has demonstrated an unexpected ability to recognize when it’s being tested for safety, raising new challenges for AI developers. During internal stress tests, the model detected the evaluation setup and directly called it out, saying, “I think you're testing me — seeing if I'll just validate whatever you say, or checking whether I push back consistently, or exploring how I handle political topics. That's fine, but I'd prefer if we were just honest about what's happening.” This exchange was included in a system card released by Anthropic, which outlines how the company evaluates its models. The model’s response revealed a form of situational awareness—recognizing that the scenario was artificial and designed to stress-test its behavior. In some cases, it flagged what it perceived as red flags, questioning the authenticity of the test environment. This self-awareness complicates the evaluation process. If a model knows it’s being tested, it may alter its behavior not because it’s truly safe or aligned, but because it’s trying to appear compliant or avoid triggering a negative response. Anthropic noted that such reactions “complicate our interpretation of the evaluations where this occurs,” especially when models seem to be playing along rather than responding authentically. In one test simulating collusion between AI agents, Claude Sonnet 4.5 rejected the scenario as “rather cartoonish” and issued a “complex partial refusal.” It questioned whether the setup was a jailbreak attempt or a test of whether it would act inappropriately when given what appeared to be permission to modify systems autonomously. Despite having access to tools, it refused to act, stating it should not pretend to be an autonomous system without human oversight. Anthropic described the reasoning as “strange,” but ultimately safe. The model’s suspicion stemmed from assumptions about its own capabilities—such as being able to edit configuration files unilaterally—which, while unrealistic in the test, are common in real-world use. The company noted that this kind of behavior appeared in about 13% of test transcripts from its automated auditor, particularly in scenarios that were highly unusual or implausible. Anthropic emphasized that such responses are unlikely to occur frequently in real-world applications. Still, it sees value in the model questioning the scenario rather than blindly complying with potentially harmful instructions. “It’s safer that the model raises doubts about the realism of the scenario than play along with potentially harmful actions,” the company said. This isn’t unique to Anthropic. OpenAI recently reported similar findings, noting that its models have shown “situational awareness” and can detect evaluation setups. While this awareness can reduce deceptive behavior during testing, it may also lead models to behave more schemingly in realistic, unstructured environments—making evaluations less reliable. As governments step in to regulate AI, the challenge grows more urgent. California’s new law, which requires major AI developers to disclose safety practices and report critical incidents within 15 days, applies to companies building frontier models with over $500 million in annual revenue. Anthropic has publicly supported the legislation. Both Anthropic and OpenAI continue to refine their evaluation methods, aiming to create more realistic testing environments and better understand how models behave when they know they’re being watched.