Code Generation On Codecontests
評価指標
Test Set pass@1
Test Set pass@5
Val Set pass@1
評価結果
このベンチマークにおける各モデルのパフォーマンス結果
比較表
| モデル名 | Test Set pass@1 | Test Set pass@5 | Val Set pass@1 |
|---|---|---|---|
| mapcoder-multi-agent-code-generation-for | 28.5 | 35.2 | 28.5 |
| planning-driven-programming-a-large-language | 34.7 | - | - |
| wizardcoder-empowering-code-large-language | 1.11 | 3.18 | 1.98 |
| motcoder-elevating-large-language-models-with | 20.77 | - | 16.72 |
| codesim-multi-agent-code-generation-and-1 | 29.1 | - | - |
| codechain-towards-modular-code-generation | 2.35 | 3.29 | 2.48 |
| motcoder-elevating-large-language-models-with | 26.34 | - | 20.35 |