Openapi Code Completion On Openapi Code
評価指標
Correctness, avg., %
Correctness, max., %
Validness, avg., %
Validness, max., %
評価結果
このベンチマークにおける各モデルのパフォーマンス結果
モデル名 | Correctness, avg., % | Correctness, max., % | Validness, avg., % | Validness, max., % | Paper Title | Repository |
|---|---|---|---|---|---|---|
| Code Llama 7B | 31.1 | 36 | 60.7 | 64 | Optimizing Large Language Models for OpenAPI Code Completion | |
| Code Llama 7B, fine-tuned with document splitting | 34 | 42 | 69.1 | 76 | Optimizing Large Language Models for OpenAPI Code Completion | |
| GitHub Copilot | 29 | 29 | 68 | 68 | Optimizing Large Language Models for OpenAPI Code Completion | |
| Code Llama 7B, fine-tuned at 4096 tokens | 32 | 45 | 63.1 | 84 | Optimizing Large Language Models for OpenAPI Code Completion |
0 of 4 row(s) selected.