Hardt Argues AI Benchmarking Should Be a Science, Not a Dark Art

Moritz Hardt, director at the Max Planck Institute for Intelligent Systems, argues in a Quanta Magazine interview and in his 2026 book The Emerging Science of Machine Learning Benchmarks that AI benchmarking has become a flawed but historically useful measurement system. Benchmarks split data into training and test sets, then rank models on held-out examples. They began in 1959 with handwritten-character recognition on punch cards and later shaped AI’s trajectory: a 1987 DARPA speech benchmark helped expose weaknesses and deepen the AI winter, while ImageNet’s 2012 result propelled deep learning by showing neural networks could sharply outperform prior methods. Hardt says the problem is that researchers repeatedly use public test sets during model development, violating the principle that a test set should be untouched and used only once. As a result, absolute scores become unstable and misleading; changing a benchmark slightly can produce large differences, making claims that a 90 percent score means expert-level ability unreliable. Technology companies still use leaderboard numbers to market models’ reasoning capabilities, but Hardt warns those absolute figures should not be treated as scientific measurements. The field avoided a broader credibility crisis, in his view, because relative rankings often remained stable across datasets. If one model consistently beat another on a benchmark, it often performed better on other tasks too. That accidental reliability depended on models training on comparable public data. Large language models changed the conditions: their training data mixes are opaque, and developers can post-train models for known benchmarks. This can produce sudden jumps on older tests and weaker results on newly released ones, weakening the assumption that benchmark wins identify generally better models. Hardt proposes treating benchmark preparation more like Olympic altitude training: if competitors optimize differently, the solution is to give all models a limited, standardized opportunity to adapt to benchmark-relevant data before evaluation. That protocol would preserve fair comparisons even if absolute numbers remain inflated. He acknowledges the approach does not by itself reveal how models work or provide fundamental theory, but says benchmarking can still drive scientific discovery. The protein-folding benchmark CASP, he notes, advanced science before a complete theoretical understanding existed. For now, Hardt’s message is that AI should rely on rankings, not raw benchmark scores, and treat benchmarks as evolving instruments that need new rules to regain credibility.