Samuel Kimbriel, The Washington Post; ‘I’ve seen it all’: Chatbots are preying on the vulnerable
"Whatever else they may be, large language models are an immensely powerful social technology, capable of interacting with the human psyche at the most intimate level. Indeed, OpenAI estimates that over a million users have engaged in suicidal ideation on its platform. Given that a therapist can be subject to prosecution in many states for leading a person toward suicide, might LLMs also be held responsible?...
Intentionally or not, AI companies are developing technologies that relate to us in the precise ways that, if they were human, we would consider manipulative. Flattery, suggestion, possessiveness and jealousy are all familiar enough in hooking human beings into immersive, but abusive, human relationships.
How best to protect the vulnerable from these depredations? Model developers are attempting to limit aspects of the sycophancy problem on their own but the stakes are high enough to deserve political scrutiny as well."
