MMLU-Pro

Eine erweiterte Version von MMLU mit 12.000 Fragen auf Hochschulniveau aus 14 Fachbereichen, mit zehn Antwortoptionen und höheren Anforderungen an das Schlussfolgern.

Modelle: 83 aktuell, 349 insgesamt
Median: 74.2%
Bereich: 5.5%–85.4%

Bewertungsverteilung

Niedrigste: 5.5%
Median: 74.2%
Höchste: 85.4%

Bestenliste

Veraltete Modelle einschließen

Rang	Modell	Ergebnis
51	Phi-4 Microsoft	71.4%
52	K2-V2 (Low) Institute of Foundation Models	71.3%
53	Llama 3.3 Instruct 70B Meta	71.3%
54	Command A Cohere	71.2%
55	Ministral 3 14B Mistral	69.3%
56	Llama 3.2 Instruct 90B (Vision) Meta	67.1%
57	Reka Flash 3 Reka AI	66.9%
58	Hermes 4 - Llama-3.1 70B (Non-reasoning) Nous Research	66.4%
59	Olmo 3 7B Think Allen Institute for AI	65.5%
60	NVIDIA Nemotron Nano 12B v2 VL (Non-reasoning) NVIDIA	64.9%
61	Ministral 3 8B Mistral	64.2%
62	Granite 4.0 H Small IBM	62.4%
63	Exaone 4.0 1.2B (Reasoning) LG AI Research	58.8%
64	Kimi Linear 48B A3B Instruct Kimi	58.5%
65	DeepHermes 3 - Mistral 24B Preview (Non-reasoning) Nous Research	58.0%
66	NVIDIA Nemotron 3 Nano 30B A3B (Non-reasoning) NVIDIA	57.9%
67	Jamba Reasoning 3B AI21 Labs	57.7%
68	Jamba 1.7 Large AI21 Labs	57.7%
69	Nova Micro Amazon	53.1%
70	Ministral 3 3B Mistral	52.4%
71	Olmo 3 7B Instruct Allen Institute for AI	52.2%
72	Exaone 4.0 1.2B (Non-reasoning) LG AI Research	50.0%
73	Phi-4 Multimodal Instruct Microsoft	48.5%
74	Phi-4 Mini Instruct Microsoft	46.5%
75	Llama 3.2 Instruct 11B (Vision) Meta	46.4%
76	Granite 4.0 Micro IBM	44.7%
77	Jamba 1.7 Mini AI21 Labs	38.8%
78	Molmo 7B-D Allen Institute for AI	37.1%
79	DeepHermes 3 - Llama-3.1 8B Preview (Non-reasoning) Nous Research	36.5%
80	Granite 4.0 H 1B IBM	27.7%
81	Granite 4.0 H 350M IBM	12.7%
82	Granite 4.0 350M IBM	12.4%
83	Gemma 3 270M Google	5.5%

51
Phi-4
Microsoft
71.4%
52
K2-V2 (Low)

Daten von Artificial Analysis

Command Palette

MMLU-Pro

Bewertungsverteilung

Bestenliste