KodusCodeReviewBench
HomeLeaderboardCompareBlog
GitHubContribute

Vendors publish claims.We publish the run.

Open the leaderboardCheck the artifacts
KodusCodeReviewBench

Benchmark

  • Leaderboard
  • Live
  • Compare models
  • Blog

Data

  • leaderboard.json
  • live.json
  • llms.txt
  • Benchmark repo

Contribute

  • Add test cases
  • AI tools benchmark

Kodus

  • kodus.io
  • GitHub
  • Discord
  • LinkedIn
  • X / Twitter

maintained by Kodus · 2026

AI code review benchmark leaderboard

30 real pull requests, 121 confirmed bugs, judged by Claude Haiku 5.5. Every model runs in two review modes inside the Kodus harness. Ranked by F2.

Why F2. F2 is the F-score with beta = 2, popularized for code review by Martian's benchmark. It weighs recall twice as much as precision, because a missed bug costs more than a comment you dismiss. F2 = 5·P·R / (4·P + R).

Review mode

The normal review, with one generalist agent.

#Model
F2
F1
Precision
Recall
Recall 95% CI
$/PR
$/bug
01Kimi K3.1 ProMoonshot63.557.249.1%68.6%59.0–78.0$1.244$0.450
02GPT-6.1 SolOpenAI45.951.062.6%43.0%32.6–54.4$0.787$0.454
03Claude Opus 5.5Anthropic43.946.852.6%42.1%30.4–54.8$0.715$0.421
04GLM-5.3Zhipu41.742.443.5%41.3%29.6–53.3$1.171$0.702
05DeepSeek V4.1 FlashDeepSeek41.041.943.4%40.5%30.5–50.0$0.134$0.082
06Claude Sonnet 5.5Anthropic40.045.057.0%37.2%28.6–46.9$0.542$0.361
07Muse Spark 1.2Meta36.435.333.6%37.2%28.0–47.2$0.725$0.483
08GLM-5.3 FlashZhipu36.337.539.8%35.5%27.7–43.2$0.098$0.069
09DeepSeek V4 ProDeepSeek34.538.447.6%32.2%22.0–42.9$0.490$0.377
10Qwen3.8 27BAlibaba30.936.451.5%28.1%19.6–36.8$0.198*$0.175
11MiniMax M3MiniMax21.926.138.1%19.8%14.4–25.8$0.154$0.193
12Gemini 3.8 FlashGoogle18.223.950.0%15.7%9.5–22.4$0.348**$0.550
13GPT-6 LunaOpenAI8.111.961.5%6.6%2.4–11.0$0.023***$0.086

*Served by AkashML in fp8 (8-bit quantized weights).

**Promotional price, valid until 2026-12-31.

***Ran at the vendor's default reasoning effort, which reasons very little on this task (about 25 reasoning tokens per PR).