Code Generation On Apps
Metrics
Competition Pass@1
Interview Pass@1
Introductory Pass@1
Results
Performance results of various models on this benchmark
Comparison Table
| Model Name | Competition Pass@1 | Interview Pass@1 | Introductory Pass@1 |
|---|---|---|---|
| motcoder-elevating-large-language-models-with | 21.18 | 32.63 | 54.26 |
| deepseek-coder-when-the-large-language-model | 11.09 | 19.70 | 33.80 |
| motcoder-elevating-large-language-models-with | 27.84 | 44.49 | 68.44 |
| codet-code-generation-with-generated-tests | - | - | 31.92 |
| measuring-coding-challenge-competence-with | 0.00% | 0.57% | 3.90% |
| codechain-towards-modular-code-generation | 2.5% | 6.4% | 29.3% |
| coderl-mastering-code-generation-through | 33.3 | 13.5 | 20 |
| coderl-mastering-code-generation-through | 0.69% | 1.80% | 6.77% |
| competition-level-code-generation-with-1 | - | - | - |
| codechain-towards-modular-code-generation | 3.75 | 7.49 | 26.29 |
| planning-driven-programming-a-large-language | 34.8 | 65.2 | 87.2 |
| competition-level-code-generation-with-1 | - | - | - |
| codesim-multi-agent-code-generation-and-1 | 0.81 | 4.21 | 26.04 |
| evaluating-large-language-models-trained-on | 0.50% | 1.00% | 5.60% |
| coderl-mastering-code-generation-through | 0.02% | 0.14% | 4.14% |
| codet-code-generation-with-generated-tests | 6.2% | 14.3% | 47.3% |
| mapcoder-multi-agent-code-generation-for | 0.00% | 0.70% | 1.30% |
| coderl-mastering-code-generation-through | 0.00% | 0.57% | 3.90% |