Beta version: *Information might not be fully accurate. Please report any discrepancies.

BigCodeBench

bigcodebenchCodingHigher is better

Next-generation HumanEval with more diverse library calls and complex tasks.

Loading chart…

Loading leaderboard…

Benchmark Info

Current SOTA

o135.5%

Best Open Source

Median Score31.1%

DistributionClustered

Score Distribution

P10

P50

P90

0100%

HumanEval

human-eval

SWE-bench Verified

swe-bench-verified

Codeforces

codeforces

LiveCodeBench v6

livecodebench-v6

LiveCodeBench Pro

livecodebench-pro

AA Coding Index

aa-coding-index