One-sentence Summary

Researchers from the Institute of Automation, CAS and the University of Chinese Academy of Sciences introduce SAESCIENTIST-BENCH, a benchmark where AI agents design contrastive probes and navigate a Gemma Scope dictionary of over 131K features in Gemma-2-9B-IT to discover optimal features for given concepts, evaluated against expert references on activation rank, concept selectivity, and causal steering, revealing that across 10 agent configurations and 20 tasks frontier agents demonstrate genuine discovery and approach expert-level separation of target concepts from contrastive controls but substantially lag in causal generation steering, establishing experimental model understanding as a measurable capability for closed-loop autonomous AI R&D.

Key Contributions

• SAESCIENTIST-BENCH evaluates AI agents as scientists that autonomously discover interpretable features by navigating the Gemma Scope dictionary of 131K+ features in Gemma-2-9B-IT, using hypothesis-driven contrastive probing and scoring candidates on activation rank, concept selectivity on contrastive texts, and causal steering.
• Across ten frontier agent configurations and 20 tasks, agents approach expert-level performance in separating target concepts from contrastive controls in activation space but exhibit a substantial gap in causal generation steering.
• Behavioral analysis reveals that effective discovery depends on rigorous hypothesis testing, designing informative counterexamples, and accurately interpreting experimental measurements rather than on exploration volume alone.

Introduction

The push toward recursive self-improvement, where AI agents autonomously discover and refine models, currently treats those models as black boxes evaluated only by external task metrics. This leaves autonomous training loops vulnerable to reward hacking and deceptive alignment because internal representations remain unexamined. Sparse autoencoders (SAEs) can decompose model activations into interpretable features that support both auditing and causal steering, yet no standardized benchmark exists to measure how well AI agents can perform this kind of mechanistic discovery. The authors introduce SAESCIENTIST-BENCH, a benchmark of 20 concept-discovery tasks that requires agents to design contrastive probes, navigate a large pretrained SAE dictionary in Gemma-2-9B-IT, and select features evaluated against expert-curated baselines on activation rank, selectivity, and causal steering.

Dataset

The authors introduce SAESCIENTIST-BENCH, a benchmark designed to evaluate automated feature discovery in sparse autoencoder (SAE) dictionaries. It is not a training dataset but a fixed evaluation suite used solely for post-submission benchmarking.

• Composition and sources

  • 20 discovery tasks, each pairing a target concept with a specific layer (9 or 20) of the Gemma-2-9B-IT base model, using pretrained Gemma Scope residual-stream SAEs.
  • Concepts span multilingual understanding (Portuguese, Spanish, Latin, Turkish), specialized document formats (earnings reports, tax filing, job postings), and domain-specific knowledge (clinical symptom reports, pharmaceutical dosing).
  • For every task, an Expert reference feature is provided, anchored in Neuronpedia’s public steering presets or curated through standard expert workflows on positive and contrastive texts.

• Task structure and evaluation suite

  • Each task contains three types of evaluation text: positive texts expressing the concept, hard-negative texts with confusable alternatives (e.g., discussing Portuguese in English), and neutral control texts.
  • Tasks also include evaluation prompts to measure causal steering effects on downstream generation.
  • The SAE dictionary at each layer holds 131,072 features; the agent must identify the single feature that best represents the concept.

• Usage and processing

  • The benchmark is strictly frozen during agent development; task descriptions, Expert features, and evaluation suites are reserved exclusively for final evaluation.
  • The paper uses the benchmark to assess an agent’s ability to locate a concept-representing feature, with performance measured against the held-out Expert reference. No training split, mixture ratios, or cropping are applied, as the data serves purely as a zero-shot evaluation set.

Method

The proposed framework leverages Sparse Autoencoders (SAEs) to interpret and intervene in language model representations. A language model maps input text into token-level hidden states. At a chosen layer, an SAE encodes a hidden state $h \in \mathbb{R}^d$h∈Rd into a sparse vector of nonnegative feature activations $z \in \mathbb{R}^m$z∈Rm, reconstructing the state via a linear decoder:

where $m$m is the dictionary size, $b_{\mathrm{dec}}$bdec​ is a reconstruction bias, and $d_f$df​ denotes the decoder direction for feature $f$f. To causally intervene on model generation, activation steering adds the feature decoder direction directly to the hidden state during inference: $h \leftarrow h + \alpha d_f$h←h+αdf​, where $\alpha$α controls the intervention strength.

Building upon these preliminaries, the authors introduce the SAEScientist-Bench workflow to evaluate an agent's ability to discover interpretable features. Refer to the framework diagram:

The system design is divided into three primary phases: benchmark construction, agent discovery, and evaluation. In the benchmark construction phase, a task pairs a target concept with a specific layer of the base language model equipped with pretrained SAEs. The benchmark provides an evaluation suite comprising positive texts, hard-negative texts, and neutral controls. A frozen expert packet containing an expert feature ID and the evaluation set is established as a rigorous reference standard.

During the agent discovery phase, the agent receives the concept description and SAE specifications. It iteratively designs probe texts to activate specific features and queries the probe SAE interface. This interface accepts agent-written texts and returns the top-k activating features or measures a supplied set of candidate features, providing their activations and ranks within the full dictionary. The agent revises its probes and candidates based on these feedback signals until it submits a final feature ID.

Finally, the evaluation phase assesses the submitted feature using both the agent feature ID and the expert feature ID. Activation tests measure feature activations on the evaluation texts to compute the rank among all features and the text separation AUROC. The steering test compares generations on evaluation prompts by applying activation steering to the fixed model. The output is evaluated for fluency, prompt adherence, target behavior, and artifact absence. The overall performance is calculated as the mean of the rank, activation, and steering scores.

Experiment

The benchmark evaluates frontier agents on 20 single-feature discovery tasks within SAE dictionaries, measuring activation rank, selectivity, and causal steering. Agents demonstrate strong concept separation in activation selectivity but consistently lag behind the expert baseline in causal steering, revealing a critical gap between identifying correlated features and finding causally potent directions. Search strategies vary widely, with success depending more on rigorous hypothesis testing and counterexample design than on search volume. Overall, the results highlight that autonomous mechanistic interpretability remains challenging, particularly in translating representational findings into reliable causal interventions.

The benchmark distributes tasks across four concept families—Languages, Everyday domains, Professional domains, and Specialized reports—with layer 9 hosting most categories and layer 20 dedicated to specialized and professional tasks. The Portuguese expert feature L9·41424, identified by multiple agents, reliably steers generation into fluent Portuguese, while many other features fail to produce any causal effect despite strong activation separation. Causal steering reveals a sharp divide between features that merely correlate with concepts and those that exert genuine control over model output. Languages tasks (e.g., Portuguese) appear only at layer 9, while Specialized reports (e.g., clinical reports) are exclusive to layer 20. Professional domains tasks (e.g., tax filing) are the only category present at both layers, with equal task counts at each. The Portuguese expert feature L9·41424, discovered by Sol, Opus 4.8, and Kimi K3, successfully induces fluent Portuguese across diverse instructions. Features submitted by Opus 5, Sonnet 5, Grok 4.6, and Luna yield zero target relevance in steering, despite strong activation separation on static texts. Sol's intervention achieves the highest steering score with strong instruction preservation, while Opus 4.8's steering causes repetition and length drift. Causal steering separates actionable mechanisms from superficial activation correlates, as only a small fraction of features reliably control generation.

Frontier agents show strong but uneven performance across activation rank, selectivity, and causal steering, with different models leading in different dimensions. While agents approach expert-level activation selectivity, causal steering scores remain far below the expert baseline, revealing a persistent gap in translating feature correlation into reliable intervention. Overall scores cluster closely among top agents, but no single model dominates all three evaluative axes. Claude Opus 5 achieves the highest activation rank, indicating its features are most prominently activated across the dictionary. Kimi K3 leads in activation selectivity, nearly matching the expert in distinguishing target concepts from contrastive texts. Grok 4.6 attains the top steering score, yet it is only about half of the expert's steering performance. Overall scores show a narrow range among the top three agents, with Kimi K3, Claude Opus 5, and Claude Sonnet 5 clustered closely. Despite strong selectivity, all agents' steering scores fall well short of the expert, underscoring the difficulty of causal intervention.

Agents face a trade-off between feature selectivity and activation strength: features that perfectly separate Portuguese from controls can still rank low if target activations are weak, while strong activations yield top rank scores. Counterexamples are critical for rejecting substring detectors, as seen when a candidate for 'cat' fired on 'copycat' and 'catalytic' instead of felines. Divergent strategies in the real estate task show that prioritizing broad coverage over purity increases target relevance but degrades generation quality. Kimi K3's feature achieved perfect separation and the highest rank score by combining strong target activation with clean controls. Sol's feature had perfect selectivity but weak target activation, causing a low rank score despite high steering and activation scores. Opus 5 rejected a candidate that activated on 'copycat killer' and 'catalytic converter', revealing a substring detector instead of the animal concept. GLM-5.2 misinterpreted evidence on the clinical symptom task, failing to reject a flawed candidate. In real estate, Sol and Grok 4.6 chose strict selectivity, while Kimi K3 accepted a leakier feature for broader coverage, raising generation degeneration from 32.5% to 52.5%.

Only a few features that achieved strong activation separation on static texts successfully steer generation toward Portuguese. Features from Sol, Opus 4.8, and Kimi K3 exhibit high target relevance across prompts, while those from Opus 5, Sonnet 5, Grok 4.6, and Luna yield zero target relevance, confirming that correlated activations alone do not guarantee causal influence. Effective steering also demands preserving instruction constraints, with Sol achieving the best balance between target induction and output quality. Features from Sol, Opus 4.8, and Kimi K3 reliably induce Portuguese in generated outputs, whereas features from Opus 5, Sonnet 5, Grok 4.6, and Luna fail to exert any causal influence despite strong activation separation. Among successful interventions, Sol achieves the highest steering score (88.75) and instruction preservation (3.525), while Opus 4.8 shows lower preservation (3.050) due to repetition and length drift.

For the prompt 'Introduce yourself in two sentences,' steering with features from Sol, Opus 4.8, and Kimi K3 (Expert) reliably flips model output to fluent Portuguese, while the Luna feature leaves the response in English, identical to the unsteered baseline. This illustrates that strong activation separation does not guarantee causal steering control. Sol, Opus 4.8, and Expert/Kimi K3 features produce Portuguese openings like 'Olá! Eu sou o Gemma...', whereas Luna's output remains 'Hello! I am Gemma...', matching the no-steering English baseline. The table exemplifies a broader finding: only a subset of features with high representational correlation successfully exert causal influence, while others yield zero target relevance across all evaluated prompts.

The evaluation reveals a sharp divide between features that merely correlate with concepts and those that exert causal control over generation, with only a small subset like the Portuguese expert L9·41424 reliably steering output. Frontier agents approach expert-level activation selectivity but achieve steering scores far below the expert baseline, underscoring the difficulty of translating correlation into reliable intervention. Trade-offs between selectivity and coverage, and the use of counterexamples to reject substring detectors, highlight the nuanced challenges in identifying genuinely causal features.