HyperAIHyperAI

Command Palette

Search for a command to run...

NVIDIA Unveils Advanced Foundation Models for Synthetic World Generation to Accelerate Physical AI Development

NVIDIA is advancing the development of physical AI through its latest updates to the NVIDIA Cosmos open world foundation models (WFMs), enabling the creation of highly realistic synthetic worlds for training robots, autonomous vehicles, and other intelligent systems. These models are part of a broader effort to address a key challenge in physical AI: the difficulty of collecting sufficient, diverse, and safe real-world data for training. Unlike large language models that can be trained on vast internet datasets, physical AI models must learn from data grounded in real-world physics and dynamics. Gathering such data at scale is time-consuming, expensive, and sometimes hazardous. Synthetic data generation offers a powerful alternative—simulating complex, varied environments with precise control over variables like lighting, weather, terrain, and object placement. NVIDIA’s updated Cosmos Predict 2.5 unifies three models—Text2World, Image2World, and Video2World—into a single lightweight architecture. This allows developers to generate consistent, multi-camera video environments from a single input, such as a text prompt, image, or video. The model supports high-fidelity, controllable world generation, making it ideal for training AI in diverse and dynamic scenarios. Cosmos Transfer 2.5 enhances this capability with high-fidelity, spatially aware style transfer. It enables developers to modify environmental conditions—such as changing weather, lighting, or terrain—across multiple cameras while preserving physical accuracy. The new version is 3.5 times smaller than its predecessor, offering faster processing, better prompt alignment, and improved physics fidelity. These foundation models integrate seamlessly into NVIDIA Isaac Sim, an open-source robotics simulation framework built on the NVIDIA Omniverse platform. Together, they support a four-part synthetic data pipeline that accelerates the development of physical AI by reducing the simulation-to-real gap. Companies across industries are already leveraging this technology. Skild AI uses Cosmos Transfer to generate diverse data variations for training general-purpose robot brains in Isaac Lab, enabling scalable testing across different robot embodiments and tasks. Serve Robotics relies on synthetic data from Isaac Sim to train its autonomous delivery robots, which have completed over 100,000 last-mile deliveries and collect nearly 170 billion image-lidar samples monthly. This data fuels continuous improvement of their AI models. Beyond logistics, Zipline uses NVIDIA Jetson platforms for drone delivery and recently received a DGX Spark AI supercomputer via drone delivery—highlighting how AI infrastructure is being deployed in innovative ways. Meanwhile, Lightwheel uses SimReady assets and synthetic datasets to help companies bridge simulation and real-world performance in robotics applications. In industrial settings, synthetic data is proving valuable for predictive maintenance. Data scientist Santiago Villa uses Omniverse and Blender to generate thousands of annotated images under varied conditions, improving boulder detection in mining operations and preventing costly downtime. FS Studio created photorealistic package variations for a global logistics partner, boosting object detection accuracy and reducing false positives. Robots for Humanity built a full simulation environment for an oil and gas client, using teleoperation to collect joint and motion data alongside synthetic imagery. Omniverse Ambassador Scott Dempsey is developing a synthetic data synthesizer that generates realistic cables from manufacturer specs, using Isaac Sim and Cosmos Transfer to create high-quality training data for cable handling and detection systems. These advancements underscore the growing role of OpenUSD and Omniverse in creating interoperable, scalable, and physically accurate digital twins. By generating synthetic worlds at scale, developers can train more robust, generalizable, and safe physical AI systems—accelerating innovation across robotics, manufacturing, transportation, and beyond.

Related Links