A team of researchers from Stanford University has uncovered serious flaws in a significant portion of AI benchmarks, raising concerns about the reliability of model evaluations across the field. In a new paper presented at NeurIPS 2025, Sanmi Koyejo, assistant professor of computer science, and doctoral student Sang Truong from the Stanford Trustworthy AI Research (STAIR) lab revealed that up to 5% of the thousands of benchmarks used to assess AI models may contain critical errors—what they call "fantastic bugs" in a nod to fictional creatures, but with very real consequences. Benchmarks are essential tools in AI development, used to measure how well models understand language, recognize images, or solve complex problems. However, the sheer volume and variety of these tests make it difficult to ensure their accuracy. The Stanford team’s analysis found that flawed benchmarks can mislead researchers, unfairly rank models, and influence major decisions on funding, development, and deployment. These "fantastic bugs" take many forms: incorrect answers, mismatched labels, ambiguous or culturally biased questions, logical inconsistencies, and even formatting issues that cause correct responses to be marked wrong. One example involved a benchmark where "$5" was the correct answer, but variations like "5 dollars" or "$5.00" were incorrectly scored as wrong—highlighting how small technical flaws can have large impacts. The consequences are far-reaching. A model might be deemed inferior due to a flawed benchmark, leading developers to abandon promising work. Conversely, underperforming models could be falsely elevated, skewing progress and misallocating resources. In one case, the model DeepSeek-R1 dropped to third-to-last place on an uncorrected benchmark but rose to second after the issues were fixed. To detect these problems, the researchers combined statistical methods from measurement theory with a large language model to identify outlier questions where many models failed unexpectedly. Their hybrid approach achieved 84% precision in flagging flawed questions across nine major benchmarks, significantly reducing the need for time-consuming manual review. The team is now collaborating with benchmark developers to correct or retire problematic tests, pushing for a shift from the current "publish-and-forget" model to one of ongoing stewardship and maintenance. While the response has been mixed, with many acknowledging the need for better standards, some remain hesitant to commit to continuous improvement. Koyejo emphasizes that as AI becomes more embedded in healthcare, education, and public services, the accuracy of benchmarking is not just a technical issue—it’s a matter of trust, safety, and progress. By improving the quality of evaluations, the team hopes to foster more reliable AI systems, better research decisions, and a stronger foundation for the future of artificial intelligence.