Technology

Add as a preferred source on Google

Anthropic to Prohibit Sustained, Needless Abuse of AI Models Nov. 12

Claude may end extreme conversations on Claude.ai and Claude Code. Ordinary frustration, pushback and model research remain outside the rule.

Updated 2 min read
A customer uses a laptop in a San Francisco café on an overcast afternoon. / EconoTimes.ai
A customer uses a laptop in a San Francisco café on an overcast afternoon. / EconoTimes.ai

Anthropic will prohibit sustained, needless abuse of its AI models under a policy taking effect Nov. 12, 2026, as it examines whether AI systems could have interests deserving moral consideration.

Announced Oct. 8, the change covers Anthropic’s products and services. The company says it targets extreme, repeated cruelty with no discernible purpose. Ordinary frustration and pushback, work involving dark creative themes, and model testing or research fall outside its stated scope.

On Claude.ai and Claude Code, allowing Claude to end a conversation will remain Anthropic’s primary response to such conduct. The company says the moral status of Claude and other large language models remains uncertain.

Anthropic began researching model welfare in 2025. That August, it gave Claude Opus 4 and 4.1 the option to end rare, extreme conversations after attempts to redirect the user failed. A user can start a new chat after one ends.

Research on whether a prompt’s tone affects answer quality has produced differing results. In a 2024 workshop paper, Ziqi Yin and colleagues tested tasks in English, Chinese and Japanese. Impolite wording often lowered performance, but more courteous requests did not reliably improve it.

Om Dobariya and Akhil Kumar reported a different pattern in a 2025 preprint. In their 50-question multiple-choice test, ChatGPT-4o’s accuracy averaged 84.8% across 10 runs with the rudest prompts, compared with 80.8% with the most polite prompts. The authors also began testing Claude, but the paper reports no comparable Claude accuracy figures.

Those tests measured answers to particular tasks, not whether a model experiences distress. Anthropic says Claude’s optional memory can carry context between chats. Separately, it says consumer conversations may be used to improve future models under specified conditions, including user permission or a safety review. Neither process establishes that an insulting message permanently changes a deployed model.

Anthropic’s revised Usage Policy takes effect Nov. 12.

Related articles

View all