One-sentence Summary

A Joint Research Team reports that GPT-6-Astra, using a unified in-context learning protocol with longer execution across 12 representative benchmarks and 8 robot configurations, achieves unprecedented embodied competence, whereas a closed-loop System 2/System 1 harness coordinating GPT-6-Astra with a low-level controller $( e . g . , \pi _ { 0 . 5 } )$(e.g.,π0.5​) is effective but costly, leaving practical embodied intelligence unresolved.

Key Contributions

• The paper presents a large-scale evaluation of GPT-6-Astra across 12 embodied benchmarks covering tabletop manipulation, humanoids, mobile platforms, and dexterous hands, using a unified protocol with in-context learning from training demonstrations and a substantially larger execution-step budget. Results show unprecedented benchmark performance and point to five research priorities: in-context learning generality, execution cost, physical-state verification, recovery from irreversible failures, and long-horizon progress preservation.

• The paper introduces and analyzes an agentic harness for embodied control in which a high-level System 2 reasoner coordinates with a low-level System 1 controller, such as π0.5, and examines action representation and chunking parameters. Experiments demonstrate that this closed-loop coordination is costly but effective for adapting the general model to robotic action interfaces.

• A real-world deployment study across 8 robot configurations identifies two gaps: trajectory demonstrations for in-context learning are expensive to collect, and trading longer execution for higher success rates is implausible under physical constraints. The paper also introduces a next-generation benchmark prototype based on three principles: tasks derive from real-world needs, no in-domain demonstrations are provided, and agents are judged under a continuously running world clock that includes deliberation time and its physical consequences.

Introduction

Embodied intelligence has traditionally depended on specialized models such as vision-language-action policies and world-action models for robotic manipulation and navigation. The emergence of GPT-6-Astra, a general-purpose model with zero-shot generalization, in-context learning, and failure recovery, changes this landscape and motivates a reassessment of what general embodied capability means for the field. Existing embodied benchmarks were largely designed for imitation-trained policies, so they often grant in-domain demonstrations and step budgets that can distort performance comparisons, while real-world deployment adds safety, cost, and time constraints. The authors evaluate GPT-6-Astra across 12 benchmarks, analyze the performance-cost trade-offs of harness design, validate their findings on real robots, and propose principles for next-generation embodied benchmarks.

Dataset

The provided text describes a qualitative embodied benchmark proof of concept rather than a conventional dataset with training splits or per-subset sizes.

Dataset composition and sources

• The environment is a fully interactive one-bedroom apartment built in NVIDIA Isaac Sim.
• It is assembled from publicly available assets and a floor plan.
• Coding agents furnish the apartment at real-world scale, with human review.
• Every part is bound to physics, including articulation, contact, position-based fluids, cloth, and everyday heating/cooking processes.

Key details per subset

• No explicit subset sizes, class distributions, or dataset statistics are reported.
• The benchmark offers two kinds of information: the household environment and tasks stated as human needs.
• Tasks are not PDDL/BDDL goal formulas; examples include making breakfast before the resident leaves.
• The benchmark provides no in-domain training episodes or scene-specific demonstrations.

Filtering and evaluation rules

• Success is judged over the whole trajectory on the world clock, not only at a final state.
• Judge predicates are defined on physical quantities kept private from the agent.
• Obligations may be unstated and can have deadlines.
• The world keeps running while the agent reasons; time between an observation and the next command is charged under the previous command, while host simulation and rendering time is not charged.

How the paper uses the data

• The environment is used for zero-shot evaluation of GPT-6-Astra controlling an AgiBot G2 through the robot’s own interface.
• The policy sees only onboard cameras and proprioception.
• The model receives no demonstrations, only a description of the embodiment.
• Training splits and mixture ratios are not applicable because this is an evaluation setting, not a training corpus.

Processing and metadata construction

• The construction pipeline is reproducible: assets and a floor plan form the home, coding agents furnish it, and human review ensures real-world scale.
• Simulation is built for causal fidelity: physical quantities such as mass, friction, and heat flow are simulated with cause and effect and grounded in real-world references.
• The text does not specify cropping strategies or metadata schema details for this proof of concept.

Method

The authors leverage in-context learning (ICL) to enable GPT-6-Astra to achieve strong performance across diverse embodied benchmarks without updating its model weights. By allowing the model to study available training data and construct a frozen knowledge package, the system can retrieve and adapt these resources to the current scene at test time. This approach yields substantial improvements, matching or exceeding the strongest published results on most settings under a standard budget. The performance comparison across 12 benchmarks is illustrated below:

The core of this methodology is the construction and utilization of a structured knowledge package. During the ICL preparation phase, GPT-6-Astra independently inspects and retains information from each benchmark's permitted training data and tool interfaces. Although the specific content varies by benchmark, the authors observe that each knowledge package consistently organizes around six recurring components. The structure of the knowledge package and its usage during test episodes are detailed in the following diagram:

These six components form a comprehensive framework for task execution. Task guidance provides written instructions detailing task goals, execution stages, and strategies, specifying subtask ordering and constraints. Motion references consist of demonstration-derived trajectories, grasp configurations, and phase anchors that offer numerical examples of action execution. Visual references include selected demonstration frames and contact crops to illustrate object appearance and intermediate task states. Executable helpers are reusable Python programs that support forward and inverse kinematics, reference retrieval, and action generation. Checks and recovery procedures define completion tests and associate failure symptoms with corrective actions. Finally, metadata and provenance document robot specifications, interface conventions, and the origin of the retained material.

During a test episode, the model reads this knowledge package alongside current observations and robot states. Rather than simply replaying demonstrations, GPT-6-Astra retrieves relevant candidate motions through the prepared helpers, adapts them to the current scene, and checks their execution against the visual references. This workflow allows the system to propose, adapt, and revise actions under feedback, effectively utilizing the stored resources for robust control.

Experiment

The experiments evaluate GPT-6-Astra across twelve embodied settings under a unified fixed-weight protocol that provides in-context learning from training demonstrations and an extended interaction budget for recovery. In-context learning yields broad gains when demonstrations cover the tested tasks, and extra interaction helps manipulation benchmarks more than long-horizon navigation, while limitations include task-specific knowledge packages, high inference cost, and failures in verification and recovery. Harness ablations confirm that closed-loop System 2 supervision and adaptive action chunking improve task performance at higher token cost, and real-world plus household validation show reliable rigid manipulation but persisting weaknesses in precise contact, deformable objects, long-horizon tasks, global scene awareness, and acting under time pressure.

On RoboDojo, the System 1-System 2 collaboration baseline outperforms VLA alone, and adding a closed-loop VLA interface improves success further while the open-loop VLA interface lowers it below both. The L=4 closed-loop variant keeps token cost near the baseline while retaining higher success, and in-context learning yields the highest success rate but uses the most tokens. Closed-loop VLA collaboration raises success rate above the baseline, whereas open-loop VLA lowers it below the baseline and below VLA alone. The L=4 closed-loop configuration reduces token cost compared with the standard closed-loop setup while retaining higher success than the baseline. In-context learning achieves the highest success rate among the reported strategies but incurs the highest token cost.

The real-world validation spans eight platforms ordered from simple to complex embodiments, covering five classes that each probe a distinct capability: single-arm grippers, dual-arm grippers, dexterous hands, wheeled mobile manipulators, and a humanoid. In this zero-shot setting the model receives structured human priors as context rather than demonstrations or a distilled knowledge package. The reported real-world episodes show that GPT-6-Astra often decides what to do correctly but struggles to act in a world that keeps moving, tending to repeat ineffective motions and rely on observations that become outdated while it thinks. The embodiment classes are ordered from simple to complex, with each class probing a different research question. Single-arm gripper platforms test whether simulation results transfer to rigid, deformable, and precise tasks such as stacking, folding, and plug insertion. Dual-arm gripper platforms probe bimanual coordination on harder zero-shot tasks like ordered folding and bimanual holding. Dexterous hands test high degree of freedom control through toy and bottle grasping. Wheeled mobile manipulators and a humanoid extend evaluation to room-scale, long-horizon tasks and combined locomotion with arm control. Qualitative real-world evidence indicates a gap between deciding what to do and acting on that decision in a continuously changing world.

On RoboDojo, the System 1-System 2 collaboration baseline outperforms VLA alone, and adding a closed-loop VLA interface further improves success while an open-loop interface degrades it; the L=4 closed-loop configuration keeps token cost near the baseline while retaining higher success, and in-context learning achieves the highest success but is the most token-intensive. Real-world zero-shot validation spans eight platforms across five embodiment classes, from single-arm and dual-arm grippers to dexterous hands, wheeled mobile manipulators, and a humanoid, using structured human priors as context rather than demonstrations. The real-world evidence indicates a gap between deciding what to do and acting in a continuously changing world, with the model often choosing correct actions but struggling to execute them effectively.