One-sentence Summary

The authors introduce BrainCause, an automated framework that combines generative and brain models to synthesize counterfactual stimuli and leverage an image-to-fMRI encoding model for targeted causal testing, thereby isolating concept-specific neural representations and guiding follow-up fMRI validation beyond conventional activation-based localization.

Key Contributions

• The paper introduces BrainCause, an automated framework that integrates generative image models with image-to-fMRI encoding models to distinguish true concept representations from correlated visual or semantic cues in fMRI data.
• The method employs a targeted causal testing pipeline that generates counterfactual image edits and correlated distractors to predict voxel-wise neural responses and isolate brain regions responding specifically to a target concept.
• The framework outputs validated candidate representations for specific concepts and proposes targeted follow-up fMRI experiments to further test or extend the discovered neural patterns.

Introduction

Understanding how the human brain encodes visual concepts is a foundational goal in neuroscience, yet traditional fMRI studies have struggled to move beyond correlational activation patterns. Prior approaches typically identify brain regions that show strong responses to a target concept using activation maximization, but these methods cannot distinguish true concept representations from responses driven by correlated visual or semantic cues like backgrounds, colors, or pose. This frequently produces false positives and leaves researchers without clear guidance for experimental validation. The authors leverage generative vision models, large language models, and image-to-fMRI encoding networks to introduce BrainCause, an automated framework that performs targeted causal testing by constructing controlled stimulus sets with counterfactual edits and semantic distractors. By isolating the target concept from co-occurring features, the framework validates genuine neural representations, filters out spurious activations, and automatically proposes informative follow-up experiments to close the loop between computational discovery and empirical validation.

Dataset

• Dataset Composition and Sources: The authors combine real and synthetic data to construct a concept-targeted image-fMRI dataset. Real neural responses are drawn from the preprocessed Natural Scenes Dataset (NSD), which contains 7T fMRI recordings from eight subjects viewing approximately 10,000 natural images each from COCO. They also build a large predicted fMRI pool using 120,000 unlabeled COCO images.

• Subset Details:

  • Positive Images: The authors generate 200 training and 100 validation images per concept using prompts from Gemma-3-27B-IT and the FLUX.2 text-to-image model. Additional positives are retrieved directly from the NSD.
  • Semantic Negative Images: The pipeline proposes 10 counter concepts per target, generates 10 prompts each, and synthesizes images with FLUX.2. After vision-language model filtering, this yields approximately 80 to 100 training and validation negatives. Measured negatives are also retrieved via a two-stage CLIP search.
  • Counterfactual Negative Images: The authors apply LLM-generated edit instructions to 50 training and 20 validation positive images, producing roughly 400 to 500 counterfactual negatives after model-based verification.

• Data Usage and Processing: The authors split the generated stimuli into explicit training and validation sets. All synthetic images are passed through an image-to-fMRI encoder to generate predicted brain responses for each subject. The authors train on a fixed mixture of 200 positives, roughly 80 to 100 semantic negatives, and 400 to 500 counterfactual negatives per concept. Retrieved real images and predicted pairs are used to validate discovered neural representations and compare against baselines like MindSimulator.

• Additional Processing and Metadata: The team precomputes CLIP embeddings for the NSD and the 120,000 external images to enable fast concept-based retrieval. A Qwen3-VL-8B vision-language model verifies concept presence or absence in both generated and retrieved images. Measured fMRI responses are standardized by normalizing each voxel across all images to a mean of zero and standard deviation of one, ensuring activation values reflect relative neural responses. Concept labels and verification tags are attached to each sample to track training splits, retrieval success rates, and coverage gaps.

Method

The BrainCause framework operates in three primary stages: causal dataset generation, concept-selective representation search, and validation with follow-up experiment design. The pipeline begins with a target concept and a subject's image–fMRI dataset, ultimately outputting a candidate voxel set and a confidence estimate for the concept's neural representation.

Refer to the framework diagram for an overview of the causal dataset generation process. Given a target concept, BrainCause constructs a dataset comprising three types of stimuli: positive images, counterfactuals, and semantic negatives. Positive images are generated to represent the target concept. Counterfactuals are created by editing the target concept out of the original image while preserving the background as much as possible, isolating the concept's visual contribution. Semantic negatives depict related but distinct concepts, designed to capture confounding semantic factors. A language model generates prompts for each stimulus type, which are used by a text-to-image model to synthesize the images. A vision-language model verifies the presence or absence of the target concept in each generated image. All images are then passed through an image-to-fMRI model to obtain predicted neural responses.

As shown in the figure below, the concept-selective representation search stage assigns each voxel three scores. The activation score measures the average response of a voxel to positive images. The semantic-negative score evaluates specificity by comparing the activation on positive images to the average activation on the top 10 semantic negative images that most strongly activate the voxel. The counterfactual score assesses specificity by comparing the activation on each positive image to the activation on its hardest edited counterpart, which is the edited version producing the highest activation for that voxel. The causal score for a voxel is the average of the semantic-negative and counterfactual scores. A candidate representation is constructed as the set of all voxels with positive causal scores or a predefined number of top-ranked voxels. This voxel set serves as the candidate region representing the concept.

The final verdict is determined by two main quantities. The first is the causal evidence for the discovered candidate representation, measured by its causal scores on both the generated evaluation dataset and the measured-data evaluation. High scores on both indicate stronger evidence that the voxel set responds selectively to the target concept rather than to correlated alternatives. The second is the concept coverage in the measured data, estimated by analyzing the fraction of desired positive and semantic-negative images that can be successfully retrieved and verified in the measured image-fMRI dataset. This reflects the informativeness of the measured data for validation. Based on the causal evidence and concept coverage, the framework returns a final decision, a candidate representation, and, when relevant, a set of informative images for follow-up experiments.

Experiment

Evaluated on the Natural Scenes Dataset across four subjects, BrainCause is benchmarked against activation-based and simulation-driven methods to validate causal region discovery, functional alignment, and cross-subject consistency. The results indicate that causal ranking effectively eliminates spurious localizations by filtering out regions driven by correlated visual factors, while isolating spatially precise areas that correspond to established functional brain networks. Additionally, the approach successfully captures fine-grained distinctions between related concepts and demonstrates reproducible cortical mappings across individuals, confirming that combining multiple causal signals produces robust and biologically faithful neural representations.

The authors compare multiple region discovery methods on fMRI data, focusing on activation and causality scores across different subjects. Results show that BrainCause achieves higher causal scores while maintaining strong activation, outperforming activation-based methods that often produce regions with high activation but low causality. The method consistently improves causal validation across subjects and demonstrates robustness in identifying concept-specific brain regions. BrainCause achieves higher causal scores compared to activation-based methods while maintaining strong activation. Activation-based methods show high activation but low causality, indicating many false positives. BrainCause consistently outperforms other methods across subjects in causal validation, demonstrating robustness and reliability.

The authors evaluate the alignment of causally discovered brain regions with known functional areas across four categories: Bodies, Faces, Places, and Words. The the the table shows that for the top 100 voxels, the regions discovered by BrainCause align closely with established functional regions, with high accuracy for Bodies, Faces, and Words. The alignment remains strong even when considering larger regions, though it decreases slightly for Places at larger voxel counts. Results show that causally discovered regions are spatially localized and consistent with known functional organization. BrainCause discovers regions that align closely with established functional areas, particularly for Bodies, Faces, and Words. Alignment accuracy remains high across different region sizes, with slight decreases for Places at larger voxel counts. Causally discovered regions are spatially localized and consistent with known functional organization across categories.

The authors compare multiple methods for discovering brain regions associated with visual concepts, focusing on activation and causality. Results show that activation-based methods achieve high activation scores but perform poorly on causal evaluation, while the proposed method, which combines activation and causal signals, maintains high activation while significantly improving causal scores across different evaluation criteria. The integration of multiple complementary signals leads to more selective and faithful region discovery. Activation-based methods achieve high activation scores but show low causal specificity, indicating many false positives. The proposed method improves causal scores substantially while maintaining comparable activation levels by combining multiple ranking signals. Combining activation and causal signals leads to more selective and reliable region discovery compared to using activation alone.

The authors compare multiple methods for discovering brain regions associated with visual concepts, focusing on activation and causality scores across different region sizes. Results show that BrainCause achieves higher causal scores while maintaining competitive activation levels, particularly improving performance on generated and measured semantic negatives and counterfactual edits. The method consistently outperforms others in causal evaluation across all tested region sizes and subjects. BrainCause achieves higher causal scores than activation-based methods while maintaining strong activation performance. BrainCause improves causal validation across generated and measured data, especially on semantic negatives and counterfactual edits. The advantage of BrainCause in causal discovery remains consistent across different region sizes and subjects.

The authors compare multiple region discovery methods on fMRI data, evaluating their ability to identify brain regions that are both strongly activated by target concepts and causally specific to them. Results show that BrainCause, which combines activation and causal signals, achieves higher causal scores while maintaining competitive activation levels compared to activation-based methods. The method produces more selective and spatially localized regions, aligning with known functional areas and demonstrating consistent performance across subjects. BrainCause achieves higher causal scores across multiple evaluation criteria while maintaining strong activation levels compared to activation-based methods. The method produces more selective and spatially localized regions, reducing false positives and aligning with established functional brain areas. BrainCause consistently outperforms other methods in causal validation across different subjects and region sizes.

The experiments evaluate BrainCause against traditional activation-based methods on fMRI data to assess their ability to identify concept-specific brain regions. By integrating both activation and causal signals, the approach validates superior causal specificity and spatial localization across diverse subjects, region sizes, and conceptual categories. Qualitatively, the method significantly reduces false positives while maintaining strong activation levels, consistently aligning discovered regions with established functional areas for visual concepts. These findings demonstrate that combining complementary signals yields more selective and reliable neuroanatomical mapping compared to relying on activation alone.