, CNN; Anthropic says you can’t be mean to Claude. It’s not about feelings
"In early October, AI giant Anthropic inserted new language into its AI usage policy.
Alongside banning users from trying to get Claude, the company’s AI system, to help them create bioweapons or sell illegal drugs, the company added a new line: Users should not “engage in sustained and needless abusive or cruel behavior toward our models.”
Claude has had the ability to end conversations with users who were “persistently harmful or abusive” since last year, but now some users could, in extreme cases, be permanently banned from using Anthropic’s AI systems entirely under the new policy. (Anthropic declined interview requests, but a spokesperson told CNN the main enforcement mechanism will still be Claude ending the conversations, and that the prohibition is not for people expressing frustration, blunt feedback, or testing and research.)
The new policy was met with some derision. Microsoft AI CEO Mustafa Suleyman responded with a melting face emoji. Some made fun of the company for treating its AI as a conscious being with feelings...
Former Reddit CEO Yishan Wong wrote on X that one reason for the prohibition is because Claude “will learn from cruelty.”
“We already know (and complain loudly about) Anthropic training its models on user inputs. Do you really want training inputs to include ‘users act cruelly for no reason and receive pleasure from it?’” he wrote.
Studies have also shown that the tone of a user’s prompts can also affect an AI’s behavior, said Professor Jindong Wang, director of the AI frontier lab at William and Mary University."
