One-sentence Summary

A team of researchers from The Chinese University of Hong Kong, the Institute of Automation (Chinese Academy of Sciences), Microsoft Research, and other institutions propose AMID, an autonomous multi-agent framework for medical imaging model development that employs Data-Conditioned Method Planning and Verification-Guided Two-Stage Optimization to produce high-performing and auditable model artifacts across 20 diverse medical imaging tasks, outperforming general-purpose machine learning engineering systems.

Key Contributions

• AMID introduces Data-Conditioned Method Planning, which refines coarse task-level search spaces into executable, parallelizable method lanes grounded in task-specific data analysis and runnable medical-imaging resources.
• The framework implements Verification-Guided Two-Stage Optimization, moving from broad exploration of diverse method lanes to selective exploitation of promising candidates while enforcing strict verification of validation protocols, metric computation, and prediction artifacts.
• Across 20 medical imaging challenge tasks, AMID outperformed evaluated general-purpose MLE systems and, on several tasks, approached or matched strong human-designed challenge solutions, indicating that domain-conditioned planning and artifact-level verification are critical for producing high-performing, auditable model artifacts.

Introduction

Large language model agents are automating machine learning engineering through iterative, feedback-driven loops, but existing systems struggle with medical imaging tasks where domain-specific constraints on preprocessing, modeling, and evaluation are critical. Generic agents often search over coarse pipelines, leading to medically inappropriate choices, and their feedback can be unreliable under expensive, delayed validation with strict patient-level splits. The authors introduce AMID, an autonomous multi-agent framework that refines task-specific method lanes via Data-Conditioned Method Planning and enforces verification of all experimental evidence through a two-stage optimization process, yielding auditable, submission-ready model artifacts.

Method

The authors leverage AMID, an autonomous medical imaging model development framework, to transform a medical-imaging task into an auditable model package. The system operates through a data-conditioned and verification-controlled agent workflow.

The workflow begins with task initialization, where the system establishes the task contract and profiles the input data. This evidence is then converted into executable method lanes via Data-Conditioned Method Planning. The planning module first builds an evidence-backed profile of the public task view, analyzing modality, geometry, and structure. It then performs resource-grounded method search by consulting local resource libraries and external foundation models to construct a hybrid method graph. The output is a portfolio of executable method lanes, such as nnU-Net segmentation, foundation model adaptation, or data augmentation strategies. Each lane contains a modeling hypothesis, required resources, and validation obligations.

Once the method lanes are established, the Self-Organizing Agent Loop coordinates the execution. A central Lifecycle Manager assigns, transfers, retires, or restarts multiple Agent Sessions. Each session operates autonomously within its own context, cycling through reflection, planning, and coding phases. These agents interact with various runtime backends and maintain a Shared Artifact Memory. This memory stores resources, skills, scores, checkpoints, notes, and reports, ensuring that evidence is not hidden in private chat histories and allowing agents to inspect previous attempts and avoid known failures.

The optimization process is governed by Verification-Guided Two-Stage Optimization. In Stage 1, Behavior-gated Exploration, worker agents are distributed across method families to submit attempts. An independent reviewer checks each submission against the active validation protocol, ensuring metric correctness and artifact validity before counting it as real lane coverage. Once sufficient coverage is achieved and plateau behavior is observed, a promotion gate selects the top candidates. In Stage 2, Selective Exploitation, the system concentrates computation on these promoted candidates. Workers refine the leading recipes through model tuning, ensemble changes, or post-processing, while repair workers fix protocol failures. A final submission protocol and agent reviewer ensure that only high-performing and fully verified model packages are accepted as the final solution.

Experiment

AMID is evaluated on the ReX-MLE benchmark of 20 medical imaging tasks under a 24-hour budget and a single GPU, outperforming prior autonomous systems on 19 tasks with the strongest gains in segmentation and detection. Its advantage stems from a structured search over method lanes, domain-specific adaptation, and systematic artifact verification, as case studies show it leverages anatomical priors, pathology foundation models, and task-specific post-processing to convert previously failed tasks into valid solutions. Failures in ultrasound enhancement and vessel classification expose remaining limitations in long-horizon image translation and coupled segmentation-classification pipelines, while the overall design demonstrates that organizing and verifying the model-building process, rather than a stronger backend alone, drives the robust competitive performance.

Across 20 medical imaging challenges, AMID with the Codex+GPT-5.5 backend produced valid results on every task and outperformed the best autonomous MLE baseline on 19 tasks, with the largest gains in segmentation and detection. It converted previously failed or near-zero segmentation runs into viable scores above 0.5 Dice and matched or exceeded human reference performance on several detection and image-quality tasks. AMID improved ISLES'22 Dice by +0.67 and SEG.A Dice by +0.89 over the strongest baseline, while raising PUMA tissue segmentation from near-zero to over 0.5 Dice. Detection gains include +0.40 AP on DENTEX and +0.46 F1 for PUMA-T1 nuclei detection. On DENTEX, NeurIPS-CellSeg, PUMA-T2-Det, and LDCT-IQA, AMID reached or surpassed the original human reference performance. AMID achieved valid accepted results for all 20 tasks, failing only to surpass the baseline on TopCoW-CTA-Cls where it tied.

Under a unified GPT-5.5 backend, AMID outperforms the best baseline agent on all three representative tasks, with the largest gain on DENTEX detection. The advantage persists across both Codex+GPT-5.5 and Claude Code runtime configurations, indicating that the system's search organization and verification, rather than the model backend alone, drive the improvements. With a shared GPT-5.5 backend, AMID achieves 0.49 AP on DENTEX, while the strongest baseline reaches only 0.08 AP. AMID's Claude Code configuration yields the highest Dice on PUMA-T1-Seg (0.64) and accuracy on TopCoW-MRA-Cls (0.50), surpassing both baselines and the Codex variant.

AMID was evaluated across 20 diverse medical imaging challenges, where it produced valid results on every task and outperformed the best autonomous MLE baseline on 19 of them, converting previously failed segmentation runs into viable scores and matching or exceeding human reference performance on several detection and image-quality tasks. Controlled experiments with a unified GPT-5.5 backend confirmed that the system's structured search organization and verification, rather than the model backend alone, drive the improvements, and the advantage persisted across different runtime configurations. The results demonstrate that AMID robustly handles tasks that baseline agents could not meaningfully address, turning near-zero scores into strong performance.