TechPrufer
Sign inGet started

Blogs›NVIDIA›World Action Models for Robot Manipulation

World Action Models for Robot Manipulation

World Action Models for Robot Manipulation

4
posts
2026

This feature thread tracks the evolution of World Action Models (WAMs) for robot manipulation, focusing on their ability to generalize beyond training data by learning world dynamics. Initial efforts focused on the limitations of Vision-Language Models (VLMs) in robotics, which excel at semantic understanding but lack predictive dynamics. WAMs, built on video world models, overcome this by learning how the world evolves, enabling better physical generalization, adaptation to new robots with fewer data, and addressing the language-to-action grounding gap. Recent advancements explore different WAM formulations, including inverse dynamics prediction, joint prediction of future states and actions, and representation-only models. Action integration methods are also evolving, moving from default action tokens to action-as-image interfaces and latent actions/plans. The popularity of WAMs has surged due to their potential to bridge the grounding gap more effectively than VLM-based approaches, with ongoing research investigating hybrid VLA/WAM models and the specific architectural components that drive performance.

2026

Beyond VLAs: How World Action Models Reshape Robot Manipulation | NVIDIA Technical Blog

8/4/2026

This post introduces the concept of World Action Models (WAMs) as a successor to Vision-Language Action (VLA) models for robot manipulation. It highlights the limitations of VLAs in physical generalization due to their focus on semantic description rather than world dynamics. WAMs, by building on video world models, learn physical dynamics, leading to improved generalization, reduced data requirements for adaptation, and better performance on new robots. The post details how the NVIDIA Cosmos 3 model, an omni-model world foundation model, provides a robust foundation for WAMs. It also presents Cosmos3-Nano-Policy-DROID and Cosmos3-Edge-Policy-DROID as examples of post-trained WAMs for robot policies, showcasing their ability to imagine while acting and their deployment considerations for workstation serving and on-device inference.

How to Evaluate General-Purpose Robot Policies for Real-World Deployment | NVIDIA Technical Blog

7/12/2026

This post introduces RoboLab, a simulation benchmarking platform designed to address shortcomings in current robotics policy evaluation. It enables robot-agnostic benchmarking with rapid, scalable task and scene generation, overcoming issues of visual/task-domain overlap, benchmark saturation, and high setup overhead. RoboLab provides advanced diagnostic tools like graded task scores, trajectory quality via SPARC, and detailed failure event logging. It also supports sensitivity analysis using Neural Posterior Estimation and competency-tagged tasks to isolate capabilities. The platform is planned for integration into NVIDIA Isaac Lab-Arena.

Develop Humanoid Robot Policies End-to-End with NVIDIA Isaac GR00T | NVIDIA Technical Blog

7/7/2026

This post introduces the NVIDIA Isaac GR00T Development Platform and its GR00T 1.7 vision-language-action (VLA) model. It details the end-to-end workflow for developing, training, evaluating, and deploying humanoid robot policies, integrating components like Isaac Lab-Arena, Isaac Teleop, GR00T 1.7, and Isaac ROS. Key technical advancements in GR00T 1.7 include pretraining on extensive real and simulated human demonstration data, adoption of the Cosmos-Reason2-2B VLM backbone for flexible resolution and native aspect ratio image encoding, full pipeline export to ONNX and TensorRT with improved reliability, enhanced long-horizon task reasoning via task/subtask decomposition, and improved benchmark performance demonstrating stronger generalization and cross-embodiment capabilities. The post also outlines a simulation workflow for a dexterous manipulation task, covering environment setup, data collection via teleoperation, data conversion to LeRobot format, post-training GR00T 1.7, and policy evaluation.

Pretrained to Imagine, Fine-Tuned to Act: The Rise of World-Action Models | NVIDIA Technical Blog

6/15/2026

This post analyzes the recent rise of World-Action Models (WAMs) in robot foundation models, contrasting them with Vision-Language-Action (VLA) models. It details the core hypotheses for WAMs, such as addressing the language-to-action grounding gap by leveraging pretrained video or world-model backbones that already model scene dynamics. The post categorizes modern WAMs based on what the model predicts (inverse dynamics, joint prediction, representation-only) and how actions are integrated (default action tokens, action as image, latent actions/plans). It also discusses architectural considerations and hypothesizes why WAMs have gained prominence recently, suggesting a potential paradigm shift towards WAMs or hybrid VLA/WAM approaches for generalist robot policies.