DeepsecBench-AA

An implementation of Vercel's DeepsecBench, a defensive cybersecurity benchmark testing how well agents find vulnerabilities in real open-source application code.

Models: 19 current, 27 in total
Median: 19.9%
Range: 1.0%–48.1%

Score distribution

Lowest: 1.0%
Median: 19.9%
Highest: 48.1%

Leaderboard

Include deprecated models

Data from Artificial Analysis

Rank	Compare	Model	Score
1		GPT-6 Sol (Daybreak Blue, max) OpenAI	48.1%
2		GPT-6 Astra (Max) OpenAI	36.9%
3		Kimi K3.1 Pro Kimi	36.7%
4		Grok 4.7 (Xhigh) SpaceXAI	26.9%
5		Claude Opus 5.5 (Max, Default Fallback) Anthropic	26.9%
6		MiMo-V2.6-Pro Xiaomi	26.2%
7		DeepSeek V4.1 Flash 1017 (Max) DeepSeek	24.9%
8		GLM-5.3 (Max) Z AI	24.3%
9		GPT-6 Luna (Max) OpenAI	23.5%
10		GLM-5.3-Flash Z AI	19.9%
11		Step 5 Preview StepFun	15.8%
12		Mistral Large 4 (High) Mistral	15.0%
13		Gemini 3.8 Flash (High) Google	14.5%
14		Qwen3.8 27B (Xhigh) Alibaba	11.5%
15		Qwen3.8 2.4T A95B Alibaba	9.5%
16		Mistral Large 4 (Medium) Mistral	8.9%
17		Beam 1.0 Reflection	6.9%
18		MiniMax-M3 MiniMax	1.1%
19		Nemotron 3 Ultra 550B A55B (Reasoning) NVIDIA	1.0%

1
GPT-6 Sol (Daybreak Blue, max)
OpenAI
48.1%
2
GPT-6 Astra (Max)