HyperAIHyperAI

Command Palette

Search for a command to run...

Google DeepMind Unveils SIMA 2, a Self-Trained AI Agent for Virtual Worlds

Google DeepMind has unveiled SIMA 2, the next generation of its generalist AI agent, marking a significant leap in AI’s ability to understand, reason, and act in complex virtual environments. Building on the 2024 debut of SIMA 1, the new model integrates Google’s Gemini 2.5 flash-lite large language model to deliver a more capable, self-improving, and human-like agent. Unlike its predecessor, which had a 31% success rate on complex tasks, SIMA 2 nearly doubles performance, achieving 65% success in known games and showing strong potential in previously unseen environments. SIMA 2 is designed as a "scalable instructable multiworld agent" that can interpret high-level instructions, break them into steps, and execute actions in 3D virtual worlds—ranging from the vast cosmos of No Man’s Sky to the chaotic physics of Goat Simulator 3. The key innovation lies in combining Gemini’s advanced language and reasoning with embodied skills learned from video game data. This allows SIMA 2 to not only follow commands but also explain its thought process. For example, when told to go to a house "the color of a ripe tomato," it reasons: "Ripe tomatoes are red, so I should go to the red house." The agent’s capabilities extend to multi-modal input, including text, voice, hand-drawn sketches, and even emojis. A simple "🪓🪵" command triggers the action of chopping down a tree, demonstrating its ability to map abstract symbols to real in-game actions. This is made possible by Gemini’s multimodal understanding and DeepMind’s method of linking symbolic input to game mechanics. One of SIMA 2’s most groundbreaking features is its self-improvement capability. While SIMA 1 relied entirely on human gameplay videos, SIMA 2 uses its initial training to generate new tasks and reward feedback through a secondary Gemini model. It then uses this self-generated data to learn from its own mistakes—mimicking how humans learn through trial and error. This closed-loop system allows the agent to adapt and improve in new, untrained environments, such as those created by DeepMind’s world model Genie 3, and shows early signs of cross-game knowledge transfer, like applying "resource gathering" skills from No Man’s Sky to MineDojo. Despite its progress, SIMA 2 is not a full solution. It still struggles with long-term planning due to limited memory, and its control of virtual input devices (like mouse and keyboard) lags behind human performance. Experts like Julian Togelius of New York University caution that the gap between virtual and real-world environments—known as the "sim-to-real gap"—remains a major hurdle. Real-world physics, sensor noise, and body dynamics are far more complex than in games. DeepMind envisions SIMA 2 as a cognitive layer for future robots, working alongside specialized motion controllers. The system’s ability to understand goals, reason, and plan could be transferred to physical robots, though this remains a long-term goal. The team has not yet announced a timeline for real-world deployment. Currently, SIMA 2 is being released as a limited research preview to select academic and developer partners. It is not a consumer product, but a tool to explore the future of AI embodiment and general intelligence. As DeepMind’s Joe Marino stated, SIMA 2 represents a "fundamental step" toward AGI and the development of general-purpose robots. The project underscores the growing trend of using diverse, large-scale virtual environments to train AI systems. While challenges remain, SIMA 2 demonstrates a powerful new direction: AI that doesn’t just follow orders, but understands, reasons, and learns on its own. It is a critical milestone in the race to build truly intelligent, adaptable systems.

Related Links