TechPrufer
Get started
Runway
Runway logoRunway · Feature trail

Robotics Policy Model

4 posts · Sep 2026

What this trail covers

Runway introduces GWM Worlds 2, an evolution of its world model for real-time environment simulation. This iteration enhances GWM Worlds by adding generated audio and rich subject/scene control, enabling interactive worlds with continuous 720p video at 24 fps and 48,000 Hz audio. It supports real-time generation, turn-based interaction, and ahead-of-time generation for applications in interactive entertainment, virtual characters, robotics, embodied-agent simulation, and generative design. The system utilizes a WorldPrompt format for defining persistent context and a timestamped event stream for actions, and is built on a finetuned autoregressive diffusion model that conditions on global context, current frame inputs, and past generated frames via a sliding KV-cache window.

All Runway trails
  1. Runway Research | Introducing GWM Worlds 2This post details the technical advancements in GWM Worlds 2, focusing on the WorldPrompt input format which splits world state into persistent context (genesis prompt, first frame) and a timestamped event stream (actions, camera input). It describes the finetuning process of a foundational audio-video generation model on this WorldPrompt format to create an autoregressive diffusion model capable of real-time generation. The model architecture is explained, highlighting its conditioning on global context, current frame inputs, and past generated frames via a sliding KV-cache window for efficient decoding. The post also discusses different usage modes (ahead-of-time, turn-based, real-time) and the technical challenges and approaches for real-time generation, including using premade prompts bound to key inputs.
  2. UndatedData Augmentation | Runway RoboticsThis post details the Data Augmentation capability for GWM-1, which generates variations of robot trajectories. It highlights video-to-video transformation (changing environments, lighting, obstacles while preserving action sequences) and image-to-video generation (creating full trajectories from a starting frame and text prompt). The post announces the availability of the Data Augmentation SDK.
  3. UndatedOffline Policy Evaluation | Runway RoboticsThis post introduces the Offline Policy Evaluation SDK for GWM-1, enabling users to simulate policy rollouts with action sequences and camera observations. It highlights the benefits of catching failures early, photorealistic simulation, and faster iteration cycles compared to physical robot testing.
  4. UndatedPolicy Model | Runway RoboticsThis post details the integration of Runway's General World Model (GWM-1) for robotics. It outlines a three-step process: fine-tuning GWM-1 on robot-specific demonstration data, receiving a custom policy model optimized for embodiment and task configuration, and deploying via a Python SDK. The SDK allows for real-time inference by feeding camera frames and proprioceptive state to the cloud API, which returns predicted action chunks. A code snippet demonstrates initializing a policy session and running an inference loop.