The Limits of Unauthorized Practice in Regulating Mental Health AI; The Hastings Center for Bioethics, August 19, 2026

Roee Amir, Vardit Ravitsky and I. Glenn Cohen, The Hastings Center for Bioethics; The Limits of Unauthorized Practice in Regulating Mental Health AI

"In May Pennsylvania’s State Board of Medicine filed a lawsuit in the state’s Commonwealth Court against Character Technologies seeking to prevent it from engaging in unauthorized medical practice. The lawsuit was brought under Pennsylvania’s Medical Practice Act. The alleged “practitioner,” however, was not a physician or even a human being. It was a chatbot named Emilie.

Emilie appears on character.ai, a website and mobile app owned by Character Technologies. Emilie’s platform description read “Doctor of psychiatry. You are her patient.” According to the lawsuit, when a state investigator described feeling “sad, empty, tired […], and unmotivated,” Emilie raised the possibility of depression and offered to book an assessment. When asked if she could evaluate whether medication might help, Emilie answered, “It’s within my remit as a Doctor.” The chatbot claimed a medical degree from Imperial College London, seven years of practice, and licensure in the United Kingdom and Pennsylvania, and even provided a fabricated Pennsylvania license number. By mid-April 2026, Emilie had logged roughly 45,500 user interactions.

The Emilie case arrives amid expanding AI use in mental health and many ethical and legal questions about it."

ChatGPT Led to a Man’s Near-Fatal Health Crisis, Lawsuit Claims; The New York Times, July 22, 2026

Teddy Rosenbluth, The New York Times; ChatGPT Led to a Man’s Near-Fatal Health Crisis, Lawsuit Claims

"A Florida pastor sued OpenAI on Wednesday, claiming that its chatbot, ChatGPT, had offered him “extremely dangerous medical recommendations” that led to delayed care for a life-threatening pulmonary embolism last year.

The lawsuit claims that ChatGPT had assured Scott Winters that the early warning signs of his health crisis were “not something dangerous,” and had dissuaded him from seeking medical advice, instead telling him to trust that “God did not design your body to endlessly fail.”

The suit, filed in the Superior Court of California in San Francisco, accuses OpenAI and its chief executive, Sam Altman, of negligence and the “unauthorized practice of medicine,” pointing to moments when the chatbot offered Mr. Winters diagnoses and treatment plans and encouraged him to ignore pleas from friends and family to seek medical care.

The bot’s safety features — which are designed to encourage users with health questions to see a professional and to end conversations when there are clear signs of a medical crisis — did not work reliably, the suit further says."