GPT-5.6 Luna vs GPT-6 Astra: Budget Code Review Misses Security Bugs
A recent benchmark evaluating AI-driven code review capabilities reveals a stark trade-off between cost efficiency and detection accuracy when comparing OpenAI’s GPT-5.6 Luna and GPT-6 Astra. Conducted in September 2026 using fifty public pull requests from repositories including Sentry, Discourse, Keycloak, Cal.com, and Grafana, the evaluation demonstrates that while the cheaper model significantly reduces operational expenses, it introduces notable precision gaps in security-sensitive contexts. The testing framework deployed identical prompts to both models, restricting output to correctness, security, concurrency, resource management, and error-handling defects. Findings were cross-verified using GPT-6 Astra and GPT-5.6 Sol as independent judges, requiring consensus for a defect to count as verified. Over the fifty test cases, GPT-6 Astra identified ninety-two verified bugs, achieving a ninety-six percent precision rate. GPT-5.6 Luna detected sixty-nine verified bugs with a seventy-four percent precision rate, meaning roughly one in four comments required manual filtering. Despite lower accuracy, Luna processed each review in twenty-three seconds compared to Astra’s thirty-six seconds, generating over three times more output tokens yet costing just $0.20 across the entire suite. Astra’s total expenditure reached $5.66. Performance divergence becomes pronounced when analyzing defect categories and repository complexity. Both models performed comparably on data and logic errors, but Luna fell behind on concurrency issues and proved substantially weaker on security vulnerabilities. The gap was most evident in Keycloak, an identity and access management platform where authentication and permission logic frequently altered. Luna validated only fifty percent of its Keycloak findings against Astra’s ninety-three percent accuracy rate, missing critical edge cases that require understanding broader authorization pathways rather than isolated code diffs. The benchmark also highlighted a fundamental limitation of diff-only review systems: models lack repository-wide context and production behavior feedback. While the two models independently caught twenty-five unique bugs, combining both identified one hundred seventeen of the one hundred forty-three total verified defects for a combined cost of $5.86. This hybrid approach suggests that engineering teams can optimize review pipelines by routing routine changes to cost-effective models while reserving higher-precision systems for critical authentication or permission modules. Reproducibility was maintained through publicly available prompts, judge verdicts, and scoring scripts. Independent reruns confirmed moderate variability in bug detection across repeated runs, reinforcing the necessity of human oversight. For organizations implementing AI code review, the findings indicate that budget-conscious deployments of lower-tier models remain viable for general correctness checks, provided teams establish strict filtering protocols and maintain dedicated scrutiny for security-critical code paths.
