HyperAI超神经

首页资讯最新论文教程数据集百科 SOTA LLM 模型天梯 GPU 天梯顶会

中文

HyperAI超神经

Code Generation On Res Q

评估指标

pass@1

评测结果

各个模型在此基准测试上的表现结果

模型名称	pass@1	Paper Title	Repository
QurrentOS-coder + Gemini 1.5 Pro	30.0	RES-Q: Evaluating Code-Editing Large Language Model Systems at the Repository Scale	-
QurrentOS-coder + Claude 3.5 Sonnet	58.0	RES-Q: Evaluating Code-Editing Large Language Model Systems at the Repository Scale	-
QurrentOS-coder + Llama 3 70b	20.0	RES-Q: Evaluating Code-Editing Large Language Model Systems at the Repository Scale	-
QurrentOS-coder + Qwen-72B-Instruct	18.0	RES-Q: Evaluating Code-Editing Large Language Model Systems at the Repository Scale	-
QurrentOS-coder + GPT-4	30.0	RES-Q: Evaluating Code-Editing Large Language Model Systems at the Repository Scale	-
QurrentOS-coder + Claude 3 Opus	36.0	RES-Q: Evaluating Code-Editing Large Language Model Systems at the Repository Scale	-
QurrentOS-coder + GPT-4o	46.0	RES-Q: Evaluating Code-Editing Large Language Model Systems at the Repository Scale	-
QurrentOS-coder + DeepSeek-Coder-V2	29.0	RES-Q: Evaluating Code-Editing Large Language Model Systems at the Repository Scale	-
QurrentOS-coder + GPT-4 Turbo	37.0	RES-Q: Evaluating Code-Editing Large Language Model Systems at the Repository Scale	-

0 of 9 row(s) selected.

Code Generation On Res Q | SOTA | HyperAI超神经