GDPval-AA v2.1

GDPval-AA v2.1 is an evaluation framework for OpenAI's GDPval dataset. It tests AI models on real-world tasks across 44 occupations and 9 major industries. Models are given shell access and web browsing capabilities in an agentic loop via Stirrup to solve tasks, with Elo ratings derived from blind pairwise comparisons.

Models: 172 current, 315 in total
Median: 1109
Range: -402–1866

Score distribution

Lowest: -402
Median: 1109
Highest: 1866

Leaderboard

Include deprecated models

Rank	Model	Score
151	gpt-oss-120b (Low) OpenAI	179
152	Magistral Medium 1.2 Mistral	176
153	K2 Think V2 Institute of Foundation Models	137
154	Qwen3 Next 80B A3B (Reasoning) Alibaba	136
155	Ministral 3 3B Mistral	37
156	Magistral Small 1.2 Mistral	12
157	LFM2.5-2.6B Liquid AI	-1
158	Gemma 4 E4B (Reasoning) Google	-29
159	Qwen3.5 2B (Reasoning) Alibaba	-39
160	NVIDIA Nemotron 3 Nano 4B NVIDIA	-51
161	Llama 4 Scout Meta	-156
162	Llama 3.3 Instruct 70B Meta	-166
163	Gemma 4 E2B (Reasoning) Google	-181
164	Llama 4 Maverick Meta	-268
165	NVIDIA Nemotron 3 Nano 30B A3B (Non-reasoning) NVIDIA	-349
166	Qwen3.5 2B (Non-reasoning) Alibaba	-359
167	MiniCPM-V 4.6 1.3B OpenBMB	-362
168	Qwen3.5 0.8B (Non-reasoning) Alibaba	-363
169	Qwen3.5 0.8B (Reasoning) Alibaba	-381
170	Nanbeige4.1-3B Nanbeige	-397
171	Phi-4 Mini Instruct Microsoft	-401
172	K2 Horizon 0.9B Institute of Foundation Models	-402

151
gpt-oss-120b (Low)
OpenAI
179
152
Magistral Medium 1.2

Data from Artificial Analysis

Command Palette

GDPval-AA v2.1

Score distribution

Leaderboard