MMMU-Pro

Un benchmark MMMU amélioré qui élimine les raccourcis et les stratégies de devinette afin de tester plus rigoureusement les modèles multimodaux dans 30 disciplines universitaires.

Modèles: 126 actuel, 303 au total
Médiane: 70.8%
Plage: 14.5%–89.2%

Distribution des scores

Le plus bas: 14.5%
Médiane: 70.8%
Le plus élevé: 89.2%

Classement

Inclure les modèles obsolètes

Rang	Modèle	Score
101	Nemotron 3 Nano Omni 30B A3B Reasoning NVIDIA	53.2%
102	Llama 4 Scout Meta	52.9%
103	NVIDIA Nemotron Nano 12B v2 VL (Reasoning) NVIDIA	52.9%
104	Qwen3.5 397B A17B (Non-reasoning) Alibaba	52.7%
105	Gemma 4 E4B (Reasoning) Google	51.4%
106	Gemma 4 E4B (Non-reasoning) Google	51.2%
107	Nova 2.0 Omni (Non-reasoning) Amazon	49.9%
108	Ministral 3 14B Mistral	49.8%
109	Nova 2.0 Lite (Non-reasoning) Amazon	49.0%
110	Mistral Small 4 (Non-reasoning) Mistral	46.5%
111	Ministral 3 8B Mistral	46.0%
112	Gemma 4 E2B (Reasoning) Google	44.6%
113	NVIDIA Nemotron Nano 12B v2 VL (Non-reasoning) NVIDIA	44.5%
114	Qwen3.5 2B (Reasoning) Alibaba	43.0%
115	Qwen3.5 2B (Non-reasoning) Alibaba	42.7%
116	Gemma 4 E2B (Non-reasoning) Google	41.8%
117	Llama 3.2 Instruct 90B (Vision) Meta	39.5%
118	Ministral 3 3B Mistral	38.1%
119	MiniCPM-V 4.6 1.3B OpenBMB	37.9%
120	Molmo2-8B Allen Institute for AI	37.5%
121	Llama 3.2 Instruct 11B (Vision) Meta	29.3%
122	LFM2.5-VL-1.6B Liquid AI	26.5%
123	Qwen3.5 0.8B (Reasoning) Alibaba	25.8%
124	Qwen3.5 0.8B (Non-reasoning) Alibaba	25.7%
125	Molmo 7B-D Allen Institute for AI	24.5%
126	Phi-4 Multimodal Instruct Microsoft	14.5%

101
Nemotron 3 Nano Omni 30B A3B Reasoning
NVIDIA
53.2%
102
Llama 4 Scout

Données de Artificial Analysis

Command Palette

MMMU-Pro

Distribution des scores

Classement