Code Generation On Pecc
평가 지표
Pass@3
평가 결과
이 벤치마크에서 각 모델의 성능 결과
모델 이름 | Pass@3 | Paper Title | Repository |
|---|---|---|---|
| Llama-3-8B-Instruct | 3.1 | PECC: Problem Extraction and Coding Challenges | - |
| Claude 3 Haiku | 27.67 | PECC: Problem Extraction and Coding Challenges | - |
| chat-bison | 8.48 | PECC: Problem Extraction and Coding Challenges | - |
| GPT-3.5 Turbo | 23.75 | PECC: Problem Extraction and Coding Challenges | - |
| WizardLM-2-7B | 3.72 | PECC: Problem Extraction and Coding Challenges | - |
| Mixtral-8x7B-Instruct | 8.35 | PECC: Problem Extraction and Coding Challenges | - |
| codechat-bison | 11.39 | PECC: Problem Extraction and Coding Challenges | - |
| Phi-3-mini-128k-instruct | 7.18 | PECC: Problem Extraction and Coding Challenges | - |
0 of 8 row(s) selected.