CritPt v2

CritPt v2 is the revised version of CritPt, a benchmark designed to test LLMs on research-level physics reasoning tasks, featuring 70 composite research challenges. Models solve problems with a code-execution tool in an offline Linux sandbox.

Intelligence Index weight: 10%
Models: 110 current, 128 in total
Median: 33.5%
Range: 0.0%–76.5%

Score distribution

Lowest: 0.0%
Median: 33.5%
Highest: 76.5%

Leaderboard

Include deprecated models

Rank	Model	Score	Cost / task	Time / task	Output tokens / task
101	K2 Horizon 7B Institute of Foundation Models	1.5%	—	—	—
102	Nemotron 3.5 Lightning NVIDIA	1.5%	$0.077	7m 29s	109K
103	DeepSeek V4.1 Flash 1017 (Non-reasoning) DeepSeek	1.0%	$0.043	2m 46s	20.8K
104	K2 Horizon 3.7B Institute of Foundation Models	1.0%	—	—	—
105	North Mini Code Cohere	1.0%	$0	5m 58s	60.8K
106	Mistral Medium 3.5 Mistral	0.5%	$0.76	8m 21s	70.2K
107	Granite 4.2 3B IBM	0.0%	—	—	12.7K
108	Granite 4.2 8B IBM	0.0%	$0.0072	3m 27s	20K
109	Mistral Small 4 (Reasoning) Mistral	0.0%	$0.0077	1m 17s	11.2K
110	NVIDIA Nemotron 3 Nano 4B NVIDIA	0.0%	—	—	10.1K

101
K2 Horizon 7B
Institute of Foundation Models
1.5%
—
—
—
102
Nemotron 3.5 Lightning

Data from Artificial Analysis

Command Palette

CritPt v2

Score distribution

Leaderboard