Accueil Actualités Articles de recherche récents Tutoriels Ensembles de données Wiki SOTA Modèles LLM Classement GPU Événements

Français

Logical Reasoning On Lingoly

Métriques

Delta_NoContext

Exact Match Accuracy

Résultats

Résultats de performance de divers modèles sur ce benchmark

Nom du modèle	Delta_NoContext	Exact Match Accuracy	Paper Title	Repository
Gemini 1.5 Pro	23.4%	32.1%	LINGOLY: A Benchmark of Olympiad-Level Linguistic Reasoning Puzzles in Low-Resource and Extinct Languages
GPT-4	21.5%	33.4%	LINGOLY: A Benchmark of Olympiad-Level Linguistic Reasoning Puzzles in Low-Resource and Extinct Languages
GPT-3.5	11.2%	21.2%	LINGOLY: A Benchmark of Olympiad-Level Linguistic Reasoning Puzzles in Low-Resource and Extinct Languages
Claude Opus	28.8%	46.3%	LINGOLY: A Benchmark of Olympiad-Level Linguistic Reasoning Puzzles in Low-Resource and Extinct Languages
Command R+	11.6%	21.5%	LINGOLY: A Benchmark of Olympiad-Level Linguistic Reasoning Puzzles in Low-Resource and Extinct Languages
Llama 3 8B	4.9%	11.4%	LINGOLY: A Benchmark of Olympiad-Level Linguistic Reasoning Puzzles in Low-Resource and Extinct Languages
Llama 3 70B	2.9%	10.3%	LINGOLY: A Benchmark of Olympiad-Level Linguistic Reasoning Puzzles in Low-Resource and Extinct Languages
Llama 2 70B	1.1%	6.4%	LINGOLY: A Benchmark of Olympiad-Level Linguistic Reasoning Puzzles in Low-Resource and Extinct Languages
GPT-4o	25.1%	37.6%	LINGOLY: A Benchmark of Olympiad-Level Linguistic Reasoning Puzzles in Low-Resource and Extinct Languages
Mixtral 8x7B	6.4%	14.2%	LINGOLY: A Benchmark of Olympiad-Level Linguistic Reasoning Puzzles in Low-Resource and Extinct Languages
Gemma 7B	2.2%	4.9%	LINGOLY: A Benchmark of Olympiad-Level Linguistic Reasoning Puzzles in Low-Resource and Extinct Languages

0 of 11 row(s) selected.