Openapi Code Completion On Openapi Code
评估指标
Correctness, avg., %
Correctness, max., %
Validness, avg., %
Validness, max., %
评测结果
各个模型在此基准测试上的表现结果
模型名称 | Correctness, avg., % | Correctness, max., % | Validness, avg., % | Validness, max., % | Paper Title | Repository |
|---|---|---|---|---|---|---|
| Code Llama 7B | 31.1 | 36 | 60.7 | 64 | Optimizing Large Language Models for OpenAPI Code Completion | |
| Code Llama 7B, fine-tuned with document splitting | 34 | 42 | 69.1 | 76 | Optimizing Large Language Models for OpenAPI Code Completion | |
| GitHub Copilot | 29 | 29 | 68 | 68 | Optimizing Large Language Models for OpenAPI Code Completion | |
| Code Llama 7B, fine-tuned at 4096 tokens | 32 | 45 | 63.1 | 84 | Optimizing Large Language Models for OpenAPI Code Completion |
0 of 4 row(s) selected.