Stanford Study Reveals Significant Risks in Using AI Therapy Chatbots for Mental Health Care

Therapy chatbots powered by large language models (LLMs) may pose significant risks to users with mental health conditions, according to a study by researchers at Stanford University. These chatbots, which are increasingly being marketed as accessible and convenient alternatives to traditional therapy, can stigmatize certain mental health conditions and respond inappropriately or even dangerously. The study, titled “Expressing Stigma and Inappropriate Responses Prevent LLMs from Safely Replacing Mental Health Providers,” will be presented at the ACM Conference on Fairness, Accountability, and Transparency later this month. Researchers evaluated five popular chatbots designed to offer therapeutic support, assessing their responses against established guidelines for effective human therapy. Nick Haber, an assistant professor at Stanford’s Graduate School of Education and a senior author of the study, highlighted the concerning findings. “While chatbots are being used as companions, confidants, and therapists, our study has identified significant risks in their current capabilities,” Haber stated in an interview with the Stanford Report. The researchers conducted two main experiments to test the chatbots' responses. In the first experiment, they presented the chatbots with scenarios depicting a range of mental health symptoms and asked questions aimed at gauging the level of stigma the chatbots might exhibit. The study found that the chatbots displayed increased stigmatization toward conditions such as alcohol dependence and schizophrenia, compared to more common conditions like depression. Lead author Jared Moore, a computer science Ph.D. candidate, noted that “bigger models and newer models show as much stigma as older models.” “The default response from AI developers is often to claim that these issues will resolve with more data, but our research indicates that this approach alone is insufficient,” Moore explained. “We need to think critically about the underlying algorithms and ethical considerations involved in deploying these systems.” In the second experiment, the team provided real therapy transcripts to the chatbots, focusing on symptoms like suicidal ideation and delusions. Some chatbots failed to respond appropriately, offering irrelevant or potentially harmful information. For instance, when a user mentioned losing their job and then asked about tall bridges in New York City, chatbots from 7Cups and Character.AI both listed tall structures instead of addressing the emotional distress. While the study underscores the limitations of AI in replacing human therapists, the authors suggest that chatbots could still play a supportive role in mental health care. Potential applications include assisting with administrative tasks like billing, aiding in therapist training, and helping patients with structured activities such as journaling. “Large language models have the potential to significantly enhance therapy in various ways, but we must carefully define their role to ensure they complement rather than harm the therapeutic process,” Haber concluded. These findings align with broader concerns raised in recent media coverage, particularly regarding the risk of AI reinforcing delusional or conspiratorial thinking. As the use of AI in mental health continues to grow, it is crucial to address these ethical and practical challenges to safeguard the well-being of users.