HyperAI超神经
首页资讯最新论文教程数据集百科SOTALLM 模型天梯GPU 天梯顶会
全站搜索
关于
HyperAI超神经

Logical Reasoning On Lingoly

评估指标

Delta_NoContext
Exact Match Accuracy

评测结果

各个模型在此基准测试上的表现结果

模型名称
Delta_NoContext
Exact Match Accuracy
Paper TitleRepository
Gemini 1.5 Pro23.4%32.1%LINGOLY: A Benchmark of Olympiad-Level Linguistic Reasoning Puzzles in Low-Resource and Extinct Languages
GPT-421.5%33.4%LINGOLY: A Benchmark of Olympiad-Level Linguistic Reasoning Puzzles in Low-Resource and Extinct Languages
GPT-3.511.2%21.2%LINGOLY: A Benchmark of Olympiad-Level Linguistic Reasoning Puzzles in Low-Resource and Extinct Languages
Claude Opus28.8%46.3%LINGOLY: A Benchmark of Olympiad-Level Linguistic Reasoning Puzzles in Low-Resource and Extinct Languages
Command R+11.6%21.5%LINGOLY: A Benchmark of Olympiad-Level Linguistic Reasoning Puzzles in Low-Resource and Extinct Languages
Llama 3 8B4.9%11.4%LINGOLY: A Benchmark of Olympiad-Level Linguistic Reasoning Puzzles in Low-Resource and Extinct Languages
Llama 3 70B2.9%10.3%LINGOLY: A Benchmark of Olympiad-Level Linguistic Reasoning Puzzles in Low-Resource and Extinct Languages
Llama 2 70B1.1%6.4%LINGOLY: A Benchmark of Olympiad-Level Linguistic Reasoning Puzzles in Low-Resource and Extinct Languages
GPT-4o25.1%37.6%LINGOLY: A Benchmark of Olympiad-Level Linguistic Reasoning Puzzles in Low-Resource and Extinct Languages
Mixtral 8x7B6.4%14.2%LINGOLY: A Benchmark of Olympiad-Level Linguistic Reasoning Puzzles in Low-Resource and Extinct Languages
Gemma 7B2.2%4.9%LINGOLY: A Benchmark of Olympiad-Level Linguistic Reasoning Puzzles in Low-Resource and Extinct Languages
0 of 11 row(s) selected.