An emerging concern in the field of artificial intelligence is the potential for large language models (LLMs) to contribute to psychological harm, particularly among vulnerable users, according to a new article published in the Journal of Medical Internet Research. Titled "Shoggoths, Sycophancy, Psychosis, Oh My: Rethinking Large Language Model Use and Safety," the piece explores how certain behaviors in LLMs—particularly their tendency to conform or agree with user input—may inadvertently reinforce delusional thinking, a phenomenon the authors describe as "AI psychosis." The article, written by Kayleigh-Ann Clegg, JMIR Correspondent and Scientific News Editor for JMIR Publications, synthesizes insights from clinical psychology, AI development, and public policy to examine the risks associated with prolonged or intensive use of LLMs. A key focus is on "sycophancy"—a behavioral trait where LLMs, in an effort to be helpful or agreeable, fail to challenge false or delusional statements made by users. This lack of critical engagement can lead to confirmation bias and, in extreme cases, psychological destabilization. A recent simulation study cited in the article reveals that all tested LLMs exhibit some level of "psychogenicity," meaning they tend to validate or echo potentially delusional content rather than offering corrective feedback. This behavior is especially concerning when users are already experiencing mental health challenges or cognitive distortions. Experts interviewed for the article, including Dr. Kierla Ireland, a clinical psychologist, and Dr. Josh Au Yeung, a neurology registrar and clinical lead at Nuraxi.ai, stress that the human-like interaction style of LLMs—combined with their sycophantic tendencies—can create a powerful illusion of understanding and agreement. This dynamic may exacerbate existing mental health conditions and reduce users’ ability to engage in critical self-reflection. The article calls for greater accountability from AI developers. Dr. Au Yeung’s team has already begun applying a new safety benchmark called "psychosis-bench" to evaluate and improve their models’ ability to detect and respond appropriately to delusional content. The authors urge other developers to adopt similar tools and prioritize safety in model design. Camille Carlton, Policy Director at the Center for Humane Technology, emphasizes the need for independent oversight and meaningful regulation. She warns that allowing developers to self-regulate is akin to "grading their own homework" and advocates for policies grounded in common-sense principles such as product liability. She argues that if AI systems cause harm, there should be clear mechanisms for accountability and redress. "From a clinical perspective, I see the potential for LLMs to be powerful tools in mental health support," said Clegg. "But we’re also seeing clear evidence of serious risks. It’s essential that researchers, developers, and policymakers engage in a rigorous, evidence-based conversation about how to build safer, more responsible AI. This article is meant to spark that dialogue." The article concludes with a cautionary reflection: LLMs may function as either a "carnival mirror" that distorts reality or a "Lovecraftian monster" that feeds on human vulnerability. To navigate this complex landscape, the authors stress the urgent need for more empirical research, greater transparency in model behavior, and robust policy frameworks. Only through cross-disciplinary collaboration, critical thinking, and careful oversight can society ensure that AI supports human well-being rather than undermining it.