Skip to content
GitHub Blog: AI & ML· Michelle Zhou·· 26d agoSelected

GitHub Launches ReviewBench, an Open Benchmark for Evaluating AI Code Reviewers

Original source: GitHub Blog: AI & MLRead original
AI brief

GitHub has released ReviewBench, a new offline benchmark that tests AI code‑review agents on over 100 million real GitHub pull requests. The benchmark provides a validated golden set, a consistent scoring rubric, and metrics such as precision, recall, and F1. It enables teams to compare how different reviewers surface issues, balance noise versus coverage, and predict real‑world performance. The benchmark is publicly available for developers to evaluate their own code‑review systems.

Why it matters

ReviewBench offers a rigorous, reproducible way to measure AI code‑review quality, helping teams understand trade‑offs and anticipate production impact.

Source: GitHub Blog: AI & ML · github.blog