One-sentence Summary

National University of Singapore, UC Berkeley, and collaborators propose Reinforcement Learning with Human-Engine Verification (RLHEV), a post-training paradigm that combines dense game-engine signals with implicit human acceptance to train an Agentic World Model (AWoMo) for scene generation, thereby providing grounded reward signals for spatial world models and improving embodied policy performance on R2R and MuJoCo.

Key Contributions

• RLHEV is a post-training paradigm that combines dense engine verification signals (collision, physics, navigability) with implicit human acceptance feedback from game development to provide grounded rewards for spatial world models.
• AWoMo is an agentic world model that proposes scene edits, observes human-engine verification, and converts accepted or repaired multimodal traces into training data.
• On UnitySceneBench, RLHEV achieves the highest score. Transfer learning yields positive signals in Unreal and Godot cross-engine experiments, and AWoMo-augmented training improves embodied policy performance on R2R, Gymnasium MuJoCo, and D4RL Gym-MuJoCo.

Introduction

Spatial intelligence tasks like video generation, 3D synthesis, and world modeling lack the scalable, reliable verification that has accelerated progress in code and reasoning agents. Prior work depends on fuzzy proxy metrics such as FVD and CLIP similarity and on expensive, low-bandwidth human ratings, which introduce noise and bias that cap post-training at imitation rather than correctness. The authors argue that game development supplies a practical recursive data engine: game engines automatically verify structural properties (collisions, navigation, script integrity) while human developers provide final acceptance judgments, yielding dense, grounded feedback. They propose the Agentic World Model (AWoMo), a developer-centered agentic workflow that captures complete world-building trajectories, and train it with Reinforcement Learning with Human-Engine Verification (RLHEV). This dual-verification loop transforms world construction into a self-improving process, giving spatial models a post-training path analogous to the one that succeeded for code.

Method

The authors propose Reinforcement Learning from Human-Engine Verification (RLHEV) to address the limitations of fuzzy, subjective reward proxies in spatial generation. Instead of relying solely on expensive and noisy human annotations for final outputs, RLHEV leverages game engines as cheap, grounded verifiers for structural properties, while reserving human judgment for final acceptance and intent alignment.

As shown in the framework diagram:

This approach shifts the paradigm from high-cost, low-quality subjective annotations to a recursive, evolving loop. The core system, named AWoMo, is organized around an omni-modal world model with four distinct interfaces. The intent interface receives task briefs and design constraints. The action interface emits scene programs, asset edits, and repair actions. The verification interface records engine checks such as collision, physics stability, and navmesh reachability. Finally, the review interface captures developer acceptance, rejection, and critique. The execution loop follows a propose, render, verify, repair, and review cycle, generating structured traces that serve as training data.

Refer to the figure below:

The bottom pipeline illustrates this human-engine feedback loop. Raw inputs are processed by an Agentic World Model, which interacts with a Game Engine and a Game Developer. The engine provides localized failure signals through a recursive feedback loop involving user intent, physics, multimodal large language models, and agents. This contrasts sharply with the top pipeline, where synthesizing and manual annotation lead to fuzzy rewards and overoptimization sinkholes.

At the heart of this architecture is a shared executable scene-program representation.

As illustrated in the figure below:

The World Model Trained by RLHEV bridges understanding and generation through this Scene Program. The generation process (Forward Map) maps text intent, design goals, and task specifications into scene synthesizing, producing entities, transforms, materials, physics, and behavior scripts. The understanding process (Inverse Map) performs scene reasoning on images, videos, and 3D Gaussian Splatting to reverse-engineer the scene program. The model is grounded by Engine-native Labels (spatial, contacts, affordances, physics) and Human World Knowledge Alignment (developer verification).

To formalize the data collection, the authors introduce the Unified World-Development Protocol (UWDP). This typed multimodal protocol converts ordinary game-development work into state-action-check-review traces. A compact instance is defined as $u_t = (b, o_t, s_t, a_t, g_t, v_t, h_t, \rho_t)$ut​=(b,ot​,st​,at​,gt​,vt​,ht​,ρt​), where $b$b is the design intent, $o_t$ot​ is the object identifier, $s_t$st​ stores spatial and semantic fields, $a_t$at​ is the edit action, $g_t$gt​ is the engine output, $v_t$vt​ is rendered evidence, $h_t$ht​ is the reviewer decision, and $\rho_t$ρt​ links repairs and risks. This protocol ensures that the workflow used to build a world simultaneously emits its own verifiable training data.

Experiment

The evaluation uses a human-engine verification workflow where a world model generates or edits game assets, receives feedback from engine checks and a human reviewer, and is post-trained on that signal. Experiments on UnitySceneBench show that combining human and engine rewards achieves the best asset classification performance, while pretraining on source data improves out-of-distribution generalization within Unity and yields positive transfer gains to Unreal and Godot engines. Embodied diagnostic runs further demonstrate that AWoMo-augmented training improves policy performance on navigation and locomotion tasks. Overall, the pilot studies support human-engine verification as a practical feedback source, with the strongest gains observed under dual verification, though larger-scale studies are needed to confirm generalization.