OpenAI has retracted its previous recommendation to adopt SWE-Bench Pro, following an independent audit that revealed approximately 30 percent of the benchmark’s tasks contain fundamental design and contamination flaws. The decision underscores growing challenges in accurately measuring artificial intelligence coding capabilities and highlights the necessity for rigorous evaluation standards in model deployment and safety frameworks. The audit was initiated after OpenAI previously directed the research community toward SWE-Bench Pro as a superior alternative to SWE-Bench Verified, which was found to suffer from similar validation issues. SWE-Bench Pro originally demonstrated significant model progress, with frontier AI systems improving their pass rate from 23.3 percent to 80.3 percent over an eight-month period. However, concerns regarding data integrity prompted OpenAI to deploy a comprehensive quality assurance pipeline. This system combined automated filtering, Codex-powered investigator agents, and a structured human review campaign involving experienced software engineers. The pipeline analyzed model attempts, task metadata, test cases, and failure traces to identify broken or misleading datapoints. The investigation classified evaluation defects into four primary categories. Human reviewers ultimately identified more issues than the automated agent pipeline, particularly flagging tasks with insufficient test coverage. OpenAI attributed the benchmark’s shortcomings to the inherent structure of open-source development workflows. Pull requests are typically designed for human collaboration, resulting in problem descriptions, reference patches, and unit tests that frequently misalign. Many tests are overly strict, validating specific implementation choices rather than establishing a consistent standard for solving coding tasks. Consequently, the benchmark fails to reliably distinguish between genuine model limitations and flawed evaluation architecture. In response to these findings, OpenAI has withdrawn its earlier endorsement of SWE-Bench Pro and advised model developers to treat results from the benchmark with caution. The organization emphasized that reliable evaluations must provide accurate signal, remain resistant to gaming, and faithfully reflect true AI capabilities. While the audit demonstrated that large language models can effectively assist in scalable data quality verification, OpenAI concluded that future benchmarks require direct curation by experienced software engineers with continuous human oversight. The findings carry significant implications for AI safety research and deployment readiness, as organizations increasingly rely on standardized coding evaluations to gauge model progress and mitigate risk. The tech community is now called to develop more robust, transparent evaluation frameworks that prioritize methodological rigor over automated scale.