OpenAI’s latest research reveals a fundamental flaw in large language models like ChatGPT: hallucinations—confidently stating false information—are not just a bug but a mathematically inevitable consequence of how these systems work. The paper provides the most rigorous explanation yet for why AI models fabricate facts, even with perfect training data, and why current fixes may be ineffective—or even counterproductive. At the core of the issue is how language models generate text: one word at a time, based on probabilistic predictions. This step-by-step process allows errors to compound. The researchers show that the total error rate in sentence generation is at least twice as high as the error rate for simple yes/no questions, because mistakes accumulate over multiple predictions. This makes hallucinations not just likely but mathematically unavoidable in many cases. The problem is further worsened by the scarcity of certain facts in training data. The paper finds that if a fact appears only once in the training set for 20% of people, the model will get at least 20% of related queries wrong. In one test, DeepSeek-V3 was asked for the birthday of Adam Kalai, one of the paper’s authors. It gave three different incorrect dates—03-07, 15-06, and 01-01—despite the correct answer being in autumn. None were close. Even more troubling is the role of evaluation benchmarks. The researchers analyzed ten major AI evaluation systems and found that nine use binary grading: a correct answer earns full points, but so does a wrong answer. Expressing uncertainty—saying “I don’t know”—earns zero points, the same as a falsehood. This creates a perverse incentive: the optimal strategy is always to guess. Mathematically, the expected score for guessing always exceeds the score for abstaining. This means that even if a model is uncertain, it will be penalized for honesty. The result? A system trained to prioritize high scores will learn to fabricate answers rather than admit ignorance. OpenAI’s proposed solution is to introduce confidence thresholds. The model would be prompted to answer only if it’s more than, say, 75% confident. Correct answers earn 1 point, wrong answers lose 3 points, and uncertainty earns 0. This framework shows that under the right conditions, models would naturally choose to say “I don’t know” when uncertain—reducing hallucinations. But this fix could be disastrous for user experience. If ChatGPT said “I don’t know” to 30% of queries—based on the paper’s analysis of factual uncertainty—users accustomed to instant, confident answers would likely abandon the system. Real-world data from an air quality monitoring project in Salt Lake City shows that users engage less with systems that flag uncertainty, even when those systems are more accurate. There’s also a major computational cost. Models that assess their own confidence must evaluate multiple possible responses and estimate reliability, requiring significantly more processing power. Methods like active learning—where the AI asks clarifying questions—can improve accuracy but multiply costs. While feasible in high-stakes domains like medical diagnostics or financial trading, where the cost of error is enormous, such systems are too expensive for mass consumer use. Until the economic incentives shift—where user trust and accuracy are valued over speed and confidence—hallucinations will persist. Consumer AI development is still driven by the need for fast, engaging responses, not truthfulness. Even if hardware becomes cheaper and more efficient, the computational overhead of uncertainty-aware models will remain higher than today’s guessing-based systems. In short, OpenAI’s research exposes a deep contradiction: the best solution to hallucinations would undermine the very user experience that powers today’s AI success. Without a fundamental shift in how AI is evaluated, rewarded, and deployed, hallucinations aren’t just likely—they’re built into the system.