ITBench-AA

An implementation of IBM's ITBench benchmark, testing AI agents on Kubernetes incident root-cause analysis from offline incident snapshots. The agent inspects alerts, events, traces, and topology and identifies the contributing-factor entities (deployments, pods, namespaces, network policies, etc.) responsible for the failure.

Models: 14 current, 45 in total
Median: 39.2%
Range: 0.6%–55.6%

Score distribution

Lowest: 0.6%
Median: 39.2%
Highest: 55.6%

Leaderboard

Include deprecated models

Rank	Model	Score
1	Step 5 Preview StepFun	55.6%
2	Gemini 3.8 Flash (High) Google	52.5%
3	GLM-5.3-Flash Z AI	51.2%
4	GPT-6 Astra (Max) OpenAI	48.6%
5	GLM-5.3 (Max) Z AI	46.1%
6	Grok 4.7 (Xhigh) SpaceXAI	42.1%
7	Qwen3.8 Max (0902) Alibaba	40.3%
8	Claude Opus 5.5 (Max, Default Fallback) Anthropic	38.2%
9	Gemma 4 31B (Reasoning) Google	37.3%
10	Qwen3.5 397B A17B (Reasoning) Alibaba	34.1%
11	Gemini 3.1 Pro Preview Google	30.3%
12	Gemma 4 26B A4B (Reasoning) Google	23.6%
13	gpt-oss-120b (High) OpenAI	5.6%
14	Llama 3.3 Instruct 70B Meta	0.6%

1
Step 5 Preview
StepFun
55.6%
2
Gemini 3.8 Flash (High)

Data from Artificial Analysis

Command Palette

ITBench-AA

Score distribution

Leaderboard