HyperAI超神経
ホームニュース最新論文チュートリアルデータセット百科事典SOTALLMモデルGPU ランキング学会
検索
サイトについて
HyperAI超神経

Logical Reasoning On Lingoly

評価指標

Delta_NoContext
Exact Match Accuracy

評価結果

このベンチマークにおける各モデルのパフォーマンス結果

モデル名
Delta_NoContext
Exact Match Accuracy
Paper TitleRepository
Gemini 1.5 Pro23.4%32.1%LINGOLY: A Benchmark of Olympiad-Level Linguistic Reasoning Puzzles in Low-Resource and Extinct Languages
GPT-421.5%33.4%LINGOLY: A Benchmark of Olympiad-Level Linguistic Reasoning Puzzles in Low-Resource and Extinct Languages
GPT-3.511.2%21.2%LINGOLY: A Benchmark of Olympiad-Level Linguistic Reasoning Puzzles in Low-Resource and Extinct Languages
Claude Opus28.8%46.3%LINGOLY: A Benchmark of Olympiad-Level Linguistic Reasoning Puzzles in Low-Resource and Extinct Languages
Command R+11.6%21.5%LINGOLY: A Benchmark of Olympiad-Level Linguistic Reasoning Puzzles in Low-Resource and Extinct Languages
Llama 3 8B4.9%11.4%LINGOLY: A Benchmark of Olympiad-Level Linguistic Reasoning Puzzles in Low-Resource and Extinct Languages
Llama 3 70B2.9%10.3%LINGOLY: A Benchmark of Olympiad-Level Linguistic Reasoning Puzzles in Low-Resource and Extinct Languages
Llama 2 70B1.1%6.4%LINGOLY: A Benchmark of Olympiad-Level Linguistic Reasoning Puzzles in Low-Resource and Extinct Languages
GPT-4o25.1%37.6%LINGOLY: A Benchmark of Olympiad-Level Linguistic Reasoning Puzzles in Low-Resource and Extinct Languages
Mixtral 8x7B6.4%14.2%LINGOLY: A Benchmark of Olympiad-Level Linguistic Reasoning Puzzles in Low-Resource and Extinct Languages
Gemma 7B2.2%4.9%LINGOLY: A Benchmark of Olympiad-Level Linguistic Reasoning Puzzles in Low-Resource and Extinct Languages
0 of 11 row(s) selected.