Project Genie World Generation with Street View Grounding

5 posts · Aug 2025 – May 2026

What this trail covers

SIMA 2 integrates Gemini models to evolve from an instruction-follower into an interactive gaming companion. It can now reason about goals, converse with users, and improve itself over time. SIMA 2 demonstrates improved generalization and reliability, understanding more complex instructions and performing better in unseen games. It can also understand multimodal prompts, different languages, and emojis. SIMA 2's self-improvement capability allows it to learn through trial-and-error and Gemini-based feedback. When combined with Genie 3, SIMA 2 can sensibly orient itself, understand user instructions, and take meaningful actions toward goals in newly generated worlds.

All DeepMind trails

May 2026Simulate real-world places with Project Genie and Street ViewIntroduced the capability to ground Project Genie's world generation in Google Street View imagery. This feature allows users to select real-world locations in the U.S. as starting points for generated environments and apply stylistic transformations. The underlying technology is Maps Imagery Grounding, which leverages Street View data to anchor virtual worlds in reality.
Jan 2026D4RT: Unified, Fast 4D Scene Reconstruction & TrackingIntroduced D4RT, a unified AI model for 4D scene reconstruction and tracking across space and time. D4RT uses a unified encoder-decoder Transformer architecture with a novel query mechanism to reconstruct dynamic scenes from 2D video. It achieves up to 300x greater efficiency than previous methods, enabling real-time applications. Capabilities include point tracking (predicting 3D trajectories even when points are not visible in all frames), point cloud reconstruction (generating 3D structure without separate camera estimation), and camera pose estimation (recovering camera trajectory by aligning 3D snapshots).
Nov 2025SIMA 2: A Gemini-Powered AI Agent for 3D Virtual WorldsSIMA 2 integrates Gemini models, enabling it to reason about goals, converse with users, and self-improve through trial-and-error and Gemini-based feedback. This leads to improved generalization and reliability, allowing it to understand more complex instructions and perform better in unseen games, including multimodal prompts, different languages, and emojis. Its self-improvement capability allows it to learn in new games exclusively through self-directed play. When combined with Genie 3, SIMA 2 can operate in newly generated worlds, demonstrating unprecedented adaptability.
Aug 2025Genie 3: A new frontier for world modelsGenie 3 represents a significant advancement in world models, enabling real-time, interactive environment generation with improved consistency and realism over Genie 2. Technical innovations focus on achieving high controllability and real-time interactivity through complex auto-regressive generation that accounts for past trajectories, and maintaining environmental consistency over several minutes via emergent capabilities. The introduction of 'promptable world events' allows for more expressive text-based interactions to modify the generated world. The post also details testing Genie 3's compatibility with embodied agents like SIMA, demonstrating its utility for training agents to achieve more complex goals due to its enhanced consistency.
Advertisement