Runway Research | Introducing GWM Worlds 2

This post details the technical advancements in GWM Worlds 2, focusing on the WorldPrompt input format which splits world state into persistent context (genesis prompt, first frame) and a timestamped event stream (actions, camera input). It describes the finetuning process of a foundational audio-video generation model on this WorldPrompt format to create an autoregressive diffusion model capable of real-time generation. The model architecture is explained, highlighting its conditioning on global context, current frame inputs, and past generated frames via a sliding KV-cache window for efficient decoding. The post also discusses different usage modes (ahead-of-time, turn-based, real-time) and the technical challenges and approaches for real-time generation, including using premade prompts bound to key inputs.