One-sentence Summary

Researchers from the National University of Singapore and University of Oxford propose OmniScientist, an end-to-end omni-modal AI scientist that couples a perception layer with three autonomous agents for ideation, experiment, and writeup while enforcing novelty, validity, provenance, and traceability checks, achieving a mean overall paper score of 6.3 across 36 real-data cases and winning 85% of head-to-head comparisons against scalar-only perception.

Key Contributions

• OmniScientist is an end-to-end, omni-modal AI scientist that conducts multidisciplinary research directly from heterogeneous raw evidence, including images, signals, audio, video, 3-D structures, trajectories, tables, formulae, and graphs.
• A multimodal perception layer and three autonomous agents for ideation, experiment, and writeup operate in a deterministic, code-controlled pipeline, with idea, rigour, and claim checks enforcing novelty screening, statistical validity, execution provenance, and numerical traceability.
• On 36 real-data cases spanning 5 discipline families, OmniScientist completes the raw-data-to-manuscript process in all cases, achieves a mean overall paper score of 6.3 with the reference reasoning backbone, and outperforms a blind precomputed-feature variant on all 7 evaluation dimensions, winning 85% of head-to-head judgments.

Introduction

Recent advances in foundation models have produced AI scientists that can automate much of the research workflow, from hypothesis generation and experiments to manuscript drafting, but these systems remain limited by how scientific evidence reaches them. They typically consume data through text, code, labels, or precomputed summaries, which can erase spatial, temporal, statistical, and procedural relations before inquiry begins. Existing multimodal scientific benchmarks also fix the observation and question in advance, while agent-based systems use perception only at isolated stages, so raw observations rarely redirect the study. The authors introduce OmniScientist, an end-to-end, omni-modal AI scientist that keeps raw evidence available throughout ideation, experimentation, and writeup, and pairs open-ended agent reasoning with code-enforced checks for novelty, statistical validity, provenance, and anti-HARKing. They evaluate the system across 36 multidisciplinary cases spanning images, waveforms, audio, video, 3D structures, trajectories, tables, formulae, and graphs.

Dataset

The authors describe a demonstration suite for evaluating a scientific AI system.

Evidence taxonomy:

• Perceptual: images, micrographs, spectra, waveforms, 3-D structures.
• Symbolic: natural language or formal notation, including documents, formulae, sequences, knowledge graphs.
• Quantitative-statistical: tables, measurements, distributions.
• Procedural: trajectories, simulations, agent traces.

Dataset suite composition:

• 5 top-level discipline categories and 36 second-level cases.
• Each case uses one real, publicly downloadable dataset with a canonical citation.
• Scale ranges from 12 symbolic-regression equations to a biomedical knowledge graph with 5 million edges.
• Modalities include images, spectra, waveforms, audio, video, 3-D structures, tables, and symbolic graphs.
• Perceptual evidence covers 28 of the 36 cases.
• The remaining 8 cases are breadth controls across symbolic, quantitative-statistical, and procedural families.

Task schema:

• Each task is a single specification file containing a dataset, scientific subject, target property, and corresponding raw data.
• Nothing else is supplied, and the methodology is left entirely to the model.

How the data is used:

• The model receives the specification file and raw data, then produces an evidence-grounded paper.
• The suite is used to evaluate cross-disciplinary capability through end-to-end runs.
• Expanding to a new discipline requires only an additional specification file, without domain-specific code changes.

Processing notes:

• Perceptual and procedural evidence can be serialized into tokens.
• The authors caution that captions and unordered scalar summaries can remove local spatial or temporal structure.
• The suite uses raw records so structural cues required for scientific conclusions remain available.

The provided excerpt does not describe filtering rules, cropping, or train/dev/test splits.

Method

The authors introduce the OmniScientist framework, an end-to-end AI scientist system designed for multidisciplinary discovery. The architecture integrates a lifecycle-wide perception layer with three autonomous agents responsible for ideation, experimentation, and writeup.

Perception Layer To function effectively across diverse disciplines, the system must perceive raw artifacts directly rather than relying on precomputed summaries. The perception layer organizes observations hierarchically, categorizing artifacts into evidence families based on the reasoning paradigms they require. Within a family, specific modalities define the representation, such as images, tables, or time-series signals. To balance thoroughness with computational efficiency, the framework prioritizes native numeric analysis, extracting properties like FFT peaks or trend points directly from raw data. Visual rendering is invoked only when spatial or structural patterns are essential, and visual perception is budget-constrained to prevent unnecessary processing.

Ideation The ideation stage requires the agent to formulate a concrete, novel, and falsifiable question answerable from the supplied data. Driven by a ReAct loop, the agent establishes grounding by inventorying materials and deciding whether to inspect raw observations. It then contextualizes findings by searching literature via OpenAlex, using Crossref as a fallback. The agent develops multiple candidate ideas, assesses novelty risk and feasibility, and selects the strongest candidate. To mitigate hallucination and overconfidence, the output passes a code-enforced check that validates structural completeness, requiring a clear research question, hypothesis, experiment sketch, and falsification criterion. The system also enforces leakage checks and visual audits to prevent methodological flaws.

Experiment During experimentation, the agent translates the finalized idea into a methodological design and implements it through iterative code generation within a controlled run_python environment. The agent operates in a continuous debugging loop, analyzing execution errors and regenerating scripts until successful. It utilizes the perception layer to inspect raw input data or verify structural patterns in generated plots. The experimental design incorporates a comprehensive suite of analyses, combining a main hypothesis test with controls such as baselines, ablation studies, or sensitivity sweeps.

Once execution concludes, a code-enforced exit check verifies result provenance and statistical validity. This rigour check confirms that the agent genuinely accessed the dataset and generated figures matching the raw execution trace. It enforces strict multiple-comparison correction to account for every test attempted during the debugging loop, preventing artificial reductions in the correction denominator.

Writeup The writeup stage adapts the structure to the specific discipline, utilizing structural specifications that fix the skeleton and length of each venue style. For instance, machine learning papers include Related Work and Limitations, while chemistry papers merge Results and Discussion. Drafting proceeds from a section-level outline into full paragraphs, where each section is expanded only from the relevant slice of the structured experiment record. This ensures methodological detail reaches the methods section and decisive numbers reach the results section.

A thesis planner selects the headline claim from supported analyses and assigns other results to supporting evidence or robustness checks. References are retrieved through the OpenAlex API. Finally, a meta-audit performs a claim check, matching reported numbers and claims against the experimental record to ensure the manuscript contains fully traced numbers and supported claims before compiling the final PDF.

Experiment

The evaluation tests the framework across 36 datasets with multiple reasoning backbones, a fixed perception model, external judges, and seven metrics covering peer-review quality, multimodal grounding, and factual accuracy. Main results show consistently high manuscript quality across disciplines and modalities, with factual accuracy ranking highest, while ablations demonstrate that direct multimodal perception outperforms a scalar-only baseline and that prior-art search, the iterative agentic loop, and provenance enforcement contribute substantially. Mechanism analyses confirm that perception changes the research trajectory rather than only improving style, and the two case studies illustrate how raw waveform and radiograph observations lead to robust, verified findings in seismology and medical imaging.

The demonstration suite is designed to evaluate omni-modal scientific discovery across five top-level discipline categories and 36 cases, each anchored by one real, publicly downloadable dataset. Its scale spans orders of magnitude, from a dozen symbolic-regression equations to a biomedical knowledge graph with five million edges. The suite covers all four evidence families and many observation modalities, including images, spectra, waveforms, audio, video, 3-D structures, trajectories, tables, sequences, fields, and graphs. The suite spans five top-level discipline categories and 36 cases, each using one real, publicly downloadable dataset. Dataset size varies across orders of magnitude, from a dozen symbolic-regression equations to a graph with five million edges. Coverage includes perceptual, symbolic, quantitative-statistical, and procedural evidence across numerous modalities, including images, spectra, waveforms, audio, video, 3-D structures, trajectories, tables, sequences, fields, and graphs.

Scientific evidence is organized into four discipline-independent families: perceptual, symbolic, quantitative-statistical, and procedural or dynamic. The perceptual family includes imagery, audio, and 3-D structure, while procedural evidence covers experimental steps, code execution, simulations, and protocols. Current AI-scientist systems primarily process text and numerical data, leaving perceptual and procedural evidence relatively unexamined. The taxonomy separates evidence requiring spatial, temporal, or cross-channel interpretation into perceptual and procedural families, distinct from symbolic and quantitative-statistical forms. Current systems process symbolic and quantitative-statistical evidence more often, while perceptual and procedural artifacts remain less examined despite being serializable. Raw perceptual and procedural relations can be lost when artifacts are reduced to captions, summaries, or scalar representations.

Across the evaluated reasoning backbones, Claude Sonnet 5, GLM-5.2, and Kimi K2.7 form a high-performing cluster with small differences in peer-review and overall quality, while Qwen3.5 backbones fall clearly behind. Factual accuracy is the strongest dimension for every listed backbone, and factual accuracy and soundness are the dimensions that most closely track backbone strength. Clarity shows the least degradation across backbones. Claude Sonnet 5, GLM-5.2, and Kimi K2.7 cluster near the top on overall and peer-review scores, while Qwen3.5 backbones trail clearly. Factual accuracy is the highest-scoring dimension across all listed backbones, and clarity remains relatively stable as backbone strength changes.

The primary backbone completed every case it attempted and maintained one of the highest composite scores. GLM 5.2 achieved a slightly higher mean score but on a much smaller set of completed runs, while several open-weight backbones attempted most of the suite yet averaged lower scores. Smaller open-weight models completed fewer cases and received the lowest mean scores. Sonnet 5 completed every attempted case and recorded a comparatively high mean composite score. Open-weight backbones generally showed lower mean composite scores, with the smallest models completing fewer cases and scoring lowest.

Aggregated backbone quality remains stable across evidence modalities and discipline families, with composite scores clustering in a narrow band. Cross-domain differences are not statistically significant under a case-level permutation test. Dashes indicate categories where a backbone produced no scored papers. Composite scores remain tightly clustered across evidence modalities and discipline families. Cross-domain differences are statistically negligible under a case-level permutation test. The highest-scoring manuscripts broadly span diverse domain categories, indicating strong generalization.

The evaluation suite tests omni-modal scientific discovery across five top-level disciplines and 36 real-data cases spanning perceptual, symbolic, quantitative-statistical, and procedural evidence modalities such as images, audio, 3-D structures, trajectories, and graphs. Comparisons of reasoning backbones indicate that Claude Sonnet 5, GLM-5.2, and Kimi K2.7 form a high-performing cluster, with factual accuracy as the strongest dimension and clarity as the most stable across backbones. The primary backbone completed every attempted case with a high composite score, while smaller open-weight models completed fewer cases and scored lower. Aggregated quality remained stable across evidence modalities and discipline families, with no statistically significant cross-domain differences, suggesting broad generalization.