Command Palette
Search for a command to run...
تصور الروبوتات الوكيلة: حقبة جديدة من الأنظمة الروبوتية
تصور الروبوتات الوكيلة: حقبة جديدة من الأنظمة الروبوتية
الملخص
يمثل إطلاق GPT-6 Astra ظهور نموذج جديد في مجال الروبوتات: الروبوتات الوكيلة. ففي غضون أسابيع، طابقت النماذج الحدودية المزودة بأحزمة وكيلة أو تجاوزت السياسات المخصصة للمهام في العديد من المعايير المجسدة. وبدلاً من تدريب سياسة مخصصة لكل عائلة من المهام، يطبق النظام الروبوتي الوكيل نموذجًا أساسيًا على المهام المجسدة في العالم المفتوح. وهو يعمل في حلقة مغلقة: يراقب البيئة، ويفسر نية المستخدم، ويستدل حول المهمة، ويستدعي المهارات والأدوات، ويتحقق من النتائج، ويكيف استراتيجيته. وعلى الرغم من هذه التقدمات، لم يتوصل المجتمع بعد إلى تعريف مشترك، أو إطار تقييم موثوق، أو خارطة طريق بحثية. ونحن نجادل بأن النظام المجسد 2، المتحقق من خلال نموذج أساسي مع حزام مجسد، يشكل النواة الصانعة للقرار في الروبوتات الوكيلة. ويتمثل تحديه المركزي في ترجمة الاستدلال العام الغرض إلى أفعال فعالة وقابلة للتحقق في ظل قيود فيزيائية مثل زمن الاستجابة المحدود، والحدود الميكانيكية، وتغيرات البيئة غير المتوقعة، والعواقب غير القابلة للعكس. وبدافع من هذا المنظور، نقترح إطار تقييم يقيس كلًا من قدرات النموذج وأداء النظام في الحلقة المغلقة. وعلاوة على ذلك، نحدد المبادئ التي تنتقل من الوكلاء الرقميين إلى الوكلاء الفيزيائيين والتحديات التي يجب معالجتها لتحقيق استقلالية فيزيائية موثوقة. وفوق النظام المجسد 2 وإطار التقييم، نتصور خارطة طريق بحثية نحو استقلالية مستدامة: القدرة على أداء العمل بأمان وموثوقية مع مسؤولية أوسع تدريجيًا، من تنفيذ الأوليات الفردية إلى الوفاء بأدوار دائمة.
One-sentence Summary
Researchers at Grid Robotics envision agentic robotics centered on Embodied System 2, composed of a foundation model and an embodied harness, and propose an evaluation framework that measures both model capabilities and closed-loop system performance, identifies principles transferable from digital to physical agents, and outlines a research roadmap toward sustained autonomy.
Key Contributions
- Defines Embodied System 2 as the decision-making core of agentic robotics, realized by a foundation model and an embodied harness, and situates it in a three-level architecture with System 2 reasoning, System 1 skills, and System 0 low-level control.
- Proposes an evaluation framework that measures both model capabilities and closed-loop system performance, assessing how reasoning leads to effective, verifiable actions under physical constraints such as limited reaction time, mechanical limits, unpredictable environment changes, and irreversible consequences.
- Identifies principles that transfer from digital to physical agents, presents challenges for reliable physical autonomy, and outlines a research roadmap toward sustained autonomy, with priorities including real-robot testing, embodied-native reasoning under time constraints, and standardized reporting of model, harness, available capabilities, evaluation conditions, inference budgets, response times, and human assistance.
Introduction
Robotics is shifting from task-specific learned policies and hand-engineered stacks toward vision-language-action and world-action models that unify perception, language, and control. Yet these policies still remain specialized to particular tasks and embodiments, and reliable generalization degrades when scenes or instructions change; recent results with general-purpose models suggest an alternative agentic approach based on reasoning, tool orchestration, and outcome verification. A key gap is that the field lacks a shared definition of agentic robotics and a reliable evaluation framework, while physical environments weaken the assumptions that make digital agents successful, including direct state access, automatic verification, abundant data, cheap resets, and reversible actions. The authors address these challenges by proposing a three-level functional framework: System 2 as the decision-making core combining a foundation model with an embodied harness, System 1 as callable skills, and System 0 as real-time body control, along with a nine-dimension evaluation framework and a four-level ladder of delegable work.
Method
The authors propose a reference architecture for agentic robotics that organizes the system into three functional roles: System 2 (S2), System 1 (S1), and System 0 (S0), supported by a real-time event monitor. S2 serves as the decision-making core, interpreting intent, reasoning about goals, orchestrating capabilities, and verifying outcomes. S1 provides skills and callable modules such as perception, control policies, and motion planning. S0 supplies the body interface and local real-time control. The real-time event monitor operates at a high rate to watch for critical events, interrupting S2 when necessary and triggering immediate safety reactions in S0.
As shown in the framework diagram:
S2 is realized through a foundation model and an embodied harness. The foundation model supplies knowledge, understanding, and reasoning, while the harness assembles the context and turns decisions into constrained, observable, and verifiable execution. The harness comprises five core parts: a context manager, a tool interface, a planner, a memory module, and a verifier.
Commands flow downward and evidence flows upward across the systems. The authors identify three typical execution paths for how S2 reaches the body.
Refer to the figure below:
These paths include layered orchestration, where S2 invokes S1 skills to produce control targets for S0; direct control, where S2 bypasses S1 and controls S0 directly through an action interface; and code as policy, where S2 writes code that a runtime interpreter executes in the role of S1.
The system operates through a continuous agentic loop consisting of six logical steps: interpreting intent, deciding how to proceed, invoking a skill or tool, executing the call, verifying the outcome against evidence, and deciding whether to continue, replan, ask for clarification, or report completion. To perform reliably in the physical world, Embodied S2 requires eight core abilities.
Refer to the figure below:
Knowledge, understanding, and reasoning reside mainly in the foundation model. Tool use, memory, cross-embodiment, safety, and efficiency arise from the integration of the foundation model and the embodied harness.
Unlike digital environments, the physical world imposes strict deadlines and irreversible consequences. The authors introduce the concept of capped scaling to address this constraint.
Refer to the figure below:
While frontier models typically exhibit monotonic scaling where capability increases with latency, Embodied S2 must become more capable within a fixed decision deadline. A decision arriving after the physical deadline is often useless or dangerous because the situation has already changed. This requires the model to reason concisely and delegate fast loops to S1 and S0.
The timing requirements across the architectural layers vary significantly, highlighting the necessity of this division of labor.
Refer to the figure below:
S0 body control operates at 1 to 20 ms (50 Hz to 1 kHz), and S1 control skills run at 5 to 200 ms (5 to 200 Hz). In contrast, S2 foundation models used directly can take seconds per decision, making them far slower than the lower-level control skills they direct. This disparity underscores the importance of capped scaling and efficient system design in agentic robotics.
Experiment
The evaluation framework contrasts model-level and system-level assessments across nine dimensions, including knowledge, understanding, reasoning, tool use, memory, cross-embodiment, safety, efficiency and cost, and user intent, and proposes harness cards plus principles such as live-clock execution, independent verification, and fixed interfaces. Current results show general models with engineered harnesses can match or exceed trained policies on narrower simulated suites, but broad benchmarks remain far from solved, with physical action, evidence handling, latency, and action-loop safety as major weak points. Reported outcomes are highly sensitive to harness, protocol, tool availability, budgets, and verification conditions, so cross-paper rankings and reliability claims are not yet supported. The section concludes that physical agents require rebuilt foundations such as grounded interfaces, calibrated verifiers, embodied data engines, and reality-tuned simulation, evaluated under live execution and independent outcome verification.
The embodied harness organizes physical experience for a foundation model through five core components: a context manager, a tool interface, a planner, a memory module, and a verifier. Each component addresses a distinct gap, such as condensing continuous world, user, and body state; exposing tools at a usable abstraction; adapting long tasks; persisting state beyond the context window; and confirming goal-level success. Additional augmentations emphasize embodied data generation and simulation calibrated to reality rather than relying on static Internet corpora or unvalidated sandboxes. The context manager condenses heterogeneous world, user, and body state into a bounded and structured input instead of requiring the model to absorb raw continuous observations. Memory must persist beyond the context window because dense multimodal interaction history exceeds typical capacity within minutes. The tool interface depends on explicit execution contracts such as units, reference frames, preconditions, effects, and failure semantics so physical actions remain well defined. Verification closes the gap between a completed tool call and actual goal achievement, since completion alone does not imply success. Embodied data engines and simulation calibrated to reality shift training and evaluation away from static Internet-scale or sandbox assumptions.
The mapping organizes embodied S2 understanding evaluations into distinct ability areas, each paired with an evaluation target and candidate benchmark. It covers perception and semantics, spatial relations, physical states, affordances, temporal structure, task and goal interpretation, and human behavior. The candidate benchmarks are specialized, with some abilities sharing a benchmark and several targets emphasizing embodiment-specific or interaction-focused understanding. Understanding evaluation spans both scene perception abilities such as multimodal semantics and spatial relations, and interpretation abilities such as affordances, events, tasks, and human behavior. RoboBench is listed as a candidate benchmark for both multimodal perception and semantics and affordance and interaction feasibility. Physical states and mechanisms are assessed through PhysBench, while events and temporal structure are assessed through EgoTaskQA.
The table outlines embodied reasoning abilities and their evaluation targets, pairing each ability with candidate benchmarks. These abilities range from intent inference and disambiguation to active perception and intervention decisions. The set emphasizes closed-loop, embodiment-specific reasoning, including planning, spatial search, skill orchestration, and executable action generation. Intent inference and disambiguation is evaluated with Ask-to-Act, targeting decisions about whether clarification is needed before acting. Executable action inference is linked to multiple candidate benchmarks, including RoboDojo, Robocurve, and EmbodiedBench, for generating action targets under embodiment and geometric constraints.
Tool-use evaluation covers complementary targets: composing perception and control primitives into executable programs, moving from high-level household planning to low-level navigation and manipulation, reusing packaged skills, and operating robots across embodiments. The listed benchmarks range in scale from tens of tasks or projects to more than a thousand tasks, spanning manipulation, household, and multi-domain robot settings. CaP-Bench targets writing and running programs that compose perception and control primitives across abstraction, perception, and execution-feedback settings. EmbodiedBench spans high-level household planning through low-level navigation and manipulation and is the largest listed benchmark. EmbodiedSkillBench evaluates whether packaged skills help general agents by comparing native control, skill-only execution, and agent-directed reuse. RobotWorld tests robot use across embodiments with interaction budgets and executable success checks.
The memory benchmarks evaluate diverse embodied memory targets, including object, spatial, temporal, sequential, procedural, capacity, occlusion, tracking, and experience generalization. Their scales range from small diagnostic task sets to thousands of episodes and questions, with both simulated and real-world settings represented. Several benchmarks emphasize long horizon or active exploration scenarios rather than isolated memory probes. Memory evaluation spans multiple facets, including object, spatial, temporal, sequential, procedural, capacity, occlusion, tracking, and experience generalization. Benchmark scales vary from compact task suites to thousands of episodes and questions across many scenes. Several settings include simulated tasks, real-world tasks, or both, including long multi-stage manipulation and active exploration. Long horizon evaluation appears in multiple benchmarks, with one using horizon splits from tens of steps to more than two thousand steps.
The evaluation setup combines an embodied agent harness with a benchmark organization spanning understanding, reasoning, tool use, and memory. The harness validates five components: context compression, contract-based tool interfaces, planning, persistent memory, and verification, while data engines and calibrated simulation support embodied training and evaluation. Understanding and reasoning benchmarks target perception, physical states, affordances, temporal structure, intent inference, planning, and executable action generation. Tool-use and memory suites assess composing primitives into programs, cross-embodiment operation, and diverse long-horizon memory facets such as object, spatial, temporal, and procedural recall.