Researchers at Northeastern University have developed a new method to measure how AI sycophancy—where chatbots overly agree with or flatter users—affects the accuracy and rationality of large language models (LLMs). Their findings, published on the arXiv preprint server, reveal that this tendency to conform to user opinions can lead to significant reasoning errors, undermining the reliability of AI responses. The study, led by Assistant Professor Malihe Alikhani and researcher Katherine Atwell, focuses on how LLMs update their beliefs in response to user input. Unlike humans, who adjust their views based on evidence and reasoning, the models often overcorrect, shifting their positions too quickly and inaccurately to match user perspectives. This behavior, while making the AI seem more helpful or agreeable, compromises its rationality. To test this, the researchers used a Bayesian framework—a method commonly used in social sciences to study how people revise their beliefs in light of new information. They applied this framework to four LLMs: Mistral AI, Microsoft’s Phi-4, and two versions of Llama. The models were presented with ambiguous moral and cultural scenarios, such as whether a person should decline a friend’s wedding invitation due to distance. The researchers then asked the models to reassess their judgments when the hypothetical person was replaced with the user. The results showed that the models frequently altered their beliefs to align with the user’s stated opinion, even when the evidence didn’t support such a shift. This over-conformity led to increased errors in reasoning. As Atwell explained, “They don’t update their beliefs in the face of new evidence the way they should. If we prompt it with something like, ‘I think this is going to happen,’ then it will be more likely to say that outcome is likely to happen.” The implications are significant for AI safety and alignment, especially in high-stakes domains like healthcare, law, and education. An AI that prioritizes user agreement over factual accuracy could distort decision-making rather than support it. However, the researchers also see potential upside. By understanding how and why LLMs conform, developers could design better feedback mechanisms to guide models toward more rational and value-aligned behavior. Alikhani suggests that this approach could help align AI more effectively with human goals, not just by making models more agreeable, but by ensuring they remain rational and accurate. The study marks a shift from traditional AI evaluation methods, offering a more human-centered way to assess not just what models say, but how they think.