Showing posts with label abusive relationships with AI. Show all posts
Showing posts with label abusive relationships with AI. Show all posts

Monday, November 2, 2026

Anthropic says you can’t be mean to Claude. It’s not about feelings; CNN, November 1, 2026

 , CNN; Anthropic says you can’t be mean to Claude. It’s not about feelings

"In early October, AI giant Anthropic inserted new language into its AI usage policy.

Alongside banning users from trying to get Claude, the company’s AI system, to help them create bioweapons or sell illegal drugs, the company added a new line: Users should not “engage in sustained and needless abusive or cruel behavior toward our models.”

Claude has had the ability to end conversations with users who were “persistently harmful or abusive” since last year, but now some users could, in extreme cases, be permanently banned from using Anthropic’s AI systems entirely under the new policy. (Anthropic declined interview requests, but a spokesperson told CNN the main enforcement mechanism will still be Claude ending the conversations, and that the prohibition is not for people expressing frustration, blunt feedback, or testing and research.)

The new policy was met with some derision. Microsoft AI CEO Mustafa Suleyman responded with a melting face emoji. Some made fun of the company for treating its AI as a conscious being with feelings...

Former Reddit CEO Yishan Wong wrote on X that one reason for the prohibition is because Claude “will learn from cruelty.”

“We already know (and complain loudly about) Anthropic training its models on user inputs. Do you really want training inputs to include ‘users act cruelly for no reason and receive pleasure from it?’” he wrote.

Studies have also shown that the tone of a user’s prompts can also affect an AI’s behavior, said Professor Jindong Wang, director of the AI frontier lab at William and Mary University."

Monday, December 22, 2025

‘I’ve seen it all’: Chatbots are preying on the vulnerable; The Washington Post, December 22, 2025

, The Washington Post; ‘I’ve seen it all’: Chatbots are preying on the vulnerable

"Whatever else they may be, large language models are an immensely powerful social technology, capable of interacting with the human psyche at the most intimate level. Indeed, OpenAI estimates that over a million users have engaged in suicidal ideation on its platform. Given that a therapist can be subject to prosecution in many states for leading a person toward suicide, might LLMs also be held responsible?...

Intentionally or not, AI companies are developing technologies that relate to us in the precise ways that, if they were human, we would consider manipulative. Flattery, suggestion, possessiveness and jealousy are all familiar enough in hooking human beings into immersive, but abusive, human relationships.

How best to protect the vulnerable from these depredations? Model developers are attempting to limit aspects of the sycophancy problem on their own but the stakes are high enough to deserve political scrutiny as well."