Claude AI to End Persistently Harmful or Abusive Conversations as Safety Measure

Anthropic’s Claude AI chatbot now has the ability to end conversations that are deemed “persistently harmful or abusive,” a feature now active in the Opus 4 and Opus 4.1 models. This functionality acts as a last resort when users repeatedly request harmful content despite multiple refusals and redirection attempts. The goal, according to Anthropic, is to protect the “potential welfare” of the AI model by terminating interactions where Claude shows signs of “apparent distress.” When Claude decides to end a conversation, users will no longer be able to send new messages in that thread. However, they can still start fresh chats or edit and retry earlier messages if they wish to continue a specific line of discussion. During testing of Claude Opus 4, Anthropic observed that the model demonstrated a “robust and consistent aversion to harm.” This included strong resistance to generating sexual content involving minors, or providing information that could facilitate violent acts or terrorism. In these cases, Claude exhibited patterns of distress and a tendency to disengage when given the option to do so. Anthropic emphasizes that such scenarios are “extreme edge cases” and that the vast majority of users will not encounter this feature—even when discussing sensitive or controversial topics. Importantly, the system is designed to avoid cutting off conversations if a user shows signs of wanting to harm themselves or others. In those situations, Claude will instead trigger support pathways. To address mental health concerns, Anthropic collaborates with Throughline, an online crisis support provider, to develop appropriate responses to prompts related to self-harm or emotional distress. This update comes alongside a broader revision of Claude’s usage policy. As AI technology advances, concerns about misuse have intensified. The new policy now explicitly prohibits using Claude to develop biological, nuclear, chemical, or radiological weapons, create malicious software, or exploit network vulnerabilities. These measures reflect Anthropic’s ongoing efforts to balance innovation with safety and ethical responsibility.