Code Generation On Bigcodebench Instruct

Pass@1

Results

Performance results of various models on this benchmark

Comparison Table

Model Name	Pass@1
bigcodebench-benchmarking-code-generation	48.2
bigcodebench-benchmarking-code-generation	51.1