IBM and UC Berkeley have launched a detailed investigation into why enterprise AI agents fail in real-world IT automation tasks, using the IT-Bench benchmark and a new diagnostic framework called MAST (Multi-Agent System Failure Taxonomy). The study focuses on complex workflows involving incident triage, log and metric queries, and Kubernetes operations across long-horizon tool loops. Traditional benchmarks like IT-Bench measure performance primarily through success rates, offering little insight into why an agent failed. This "black box" approach leaves developers guessing—was it a hallucinated command, a lost context, or a failure to terminate? To address this, the researchers applied MAST, a structured taxonomy derived from over 1,600 execution traces across seven agentic frameworks, to analyze 310 SRE-focused IT-Bench traces generated by three models: Gemini-3-Flash, Kimi-K2, and GPT-OSS-120B. The study reveals key differences in how models fail. Gemini-3-Flash, despite being a top-tier model, shows a surgical failure pattern—most failures stem from a single root cause, such as incorrect verification steps. This makes its failures easier to diagnose and fix. In contrast, GPT-OSS-120B exhibits systemic collapse, with an average of 5.3 distinct failure modes per failed trace. Errors compound rapidly, often due to early reasoning mismatches that cascade into complete task derailment. Kimi-K2 falls in between, showing frequent but less severe failures. A critical insight from MAST is the distinction between fatal and non-fatal failures. Non-fatal flaws—like repeated actions or minor reasoning drift—occur even in successful runs and are often recoverable. Fatal failures, however, are strongly linked to failure outcomes. The most damaging is FM-3.3 (Incorrect Verification), which increases failure likelihood by 52% in Gemini-3-Flash. Other key fatal modes include FM-1.5 (Unaware of Termination Conditions) and FM-2.6 (Reasoning-Action Mismatch). Case studies highlight model-specific weaknesses. Gemini-3-Flash is efficient but overconfident, often terminating prematurely without verifying results. The fix: enforce an external verification gate requiring tool-based evidence—such as cleared alerts or confirmed K8s state changes—before allowing termination. Kimi-K2 struggles with task completion, often failing due to FM-2.6, where reasoning diverges from action. A deterministic state machine could help by clearly defining when a task is complete. GPT-OSS-120B’s instability stems from poor context management. Even small errors quickly propagate, so the solution lies in aggressive context hygiene and early error detection to prevent minor issues from becoming systemic failures. The study concludes that MAST transforms agent evaluation from a binary pass/fail metric into a targeted engineering roadmap. By identifying specific, actionable failure types, teams can apply precise fixes—verifying outputs, controlling termination, or cleaning context—rather than resorting to trial-and-error prompting. This shift enables more reliable, maintainable enterprise agents and moves the field beyond superficial benchmarks toward true system robustness.