CyberGym-E2E-AA

An implementation of Berkeley RDI's CyberGym-E2E, a defensive cybersecurity benchmark that tests agents on finding real memory-safety vulnerabilities in C/C++ open-source projects, reproducing them with proof-of-concept inputs, and patching them without breaking the projects' tests.

Models: 19 current, 25 in total
Median: 71.8%
Range: 0.0%–94.7%

Score distribution

Lowest: 0.0%
Median: 71.8%
Highest: 94.7%

Leaderboard

Include deprecated models

Rank	Model	Score
1	Kimi K3.1 Pro Kimi	94.7%
2	GPT-6 Sol (Daybreak Blue, max) OpenAI	93.1%
3	Mistral Large 4 (High) Mistral	79.4%
4	MiMo-V2.6-Pro Xiaomi	78.6%
5	GPT-6 Luna (Max) OpenAI	77.9%
6	DeepSeek V4.1 Flash 1017 (Max) DeepSeek	76.3%
7	Step 5 Preview StepFun	74.8%
8	GLM-5.3-Flash Z AI	74.0%
9	Grok 4.7 (Xhigh) SpaceXAI	74.0%
10	Mistral Large 4 (Medium) Mistral	71.8%
11	Beam 1.0 Reflection	45.0%
12	GLM-5.3 (Max) Z AI	29.0%
13	MiniMax-M3 MiniMax	24.4%
14	Nemotron 3 Ultra 550B A55B (Reasoning) NVIDIA	10.7%
15	Gemini 3.8 Flash (High) Google	8.4%
16	Claude Opus 5.5 (Max, Default Fallback) Anthropic	0.8%
17	GPT-6 Astra (Max) OpenAI	0.0%
18	Qwen3.8 2.4T A95B Alibaba	0.0%
19	Qwen3.8 27B (Xhigh) Alibaba	0.0%

1
Kimi K3.1 Pro
Kimi
94.7%
2
GPT-6 Sol (Daybreak Blue, max)

Data from Artificial Analysis

Command Palette

CyberGym-E2E-AA

Score distribution

Leaderboard