TechPrufer
Get started
Google
Google logoGoogle · Feature trail

Gemini-powered XR Development

3 posts · Nov 2024 – Mar 2026

What this trail covers

The Gemini API continues to evolve, offering developers access to advanced AI models like Gemini 1.5 Flash and Veo 3 for various applications. Veo 3, a text-to-video and image-to-video model with native audio generation, is now available in paid preview via the Gemini API and Vertex AI. It supports synchronized sound, cinematic quality video generation, and realistic physics simulation. Developers can experiment with Veo 3 in Google AI Studio, with pricing at $0.75 per second for video and audio output. Videos generated by Veo 3 include SynthID watermarks. Gemini subscribers also have access in the Gemini app and Flow.

All Google trails
  1. Jump to play: Building with Gemini & MediaPipe- Google Developers BlogThis post introduces the integration of Gemini with MediaPipe to build interactive applications and games. It showcases examples like a motion-controlled Chrome Dino game using MediaPipe Pose Landmarker, a hair recoloring app using MediaPipe Image Segmenter, and several other games utilizing MediaPipe's face landmarks, gesture recognition, and face detection capabilities. The post highlights the use of Google AI Studio for rapid development and iterative refinement of these applications, emphasizing MediaPipe's on-device ML processing for low-latency, real-time interactions.
  2. Turn creative prompts into interactive XR experiences with Gemini- Google Developers BlogIntroduces the use of Gemini's Canvas feature and XR Blocks to rapidly prototype and build interactive XR experiences. Demonstrates an immersive biology simulation built with Gemini, WebGL, Three.js, and WebXR APIs. Details a workflow for setting up an 'XR Gem' with Gemini, launching experiences via Chrome on Galaxy XR, and iterating on creations. Highlights starter prompts and advanced challenges for building XR content.
  3. Veo 3 and Veo 3 Fast – new pricing, new configurations and better resolution- Google Developers BlogThis post announces the general availability and significant pricing reduction for Veo 3 and Veo 3 Fast, making them production-ready. It introduces support for 9:16 aspect ratio and 1080p HD output, detailing the parameters to control these features. The post also highlights developer use cases and provides a Python code example for generating videos.
  4. Build with Veo 3, now available in the Gemini API- Google Developers BlogIntroduces Veo 3, a new text-to-video and image-to-video model with native audio generation, to the Gemini API and Vertex AI. Details its capabilities (synchronized sound, cinematic quality, realistic physics) and provides a Python code example for video generation. Mentions pricing and SynthID watermarking. Links to documentation, cookbook, and starter app.
  5. Gemini 2.5 for robotics and embodied intelligence- Google Developers BlogThis post details how Gemini 2.5 models, including Gemini Robotics-ER, can be used for robotics applications. It provides practical examples and prompts for semantic scene understanding (object identification, gauge reading), tracking multiple objects, detecting open-ended concepts like 'spill', and generating robot arm trajectories. It also showcases code generation for pick-and-place tasks using a robot control API, demonstrating different strategies and the ability to adapt plans based on reachability constraints. The post highlights the integration of spatial reasoning with code generation for zero-shot task execution.
  7. Google AI for game developers- Google Developers BlogThis post announces the Gemma 3 model, highlighting its multimodal input, extended context, function calling, and 140+ language support. It introduces the Gemma Unity plugin, built on Gemma.cpp for on-device inference, and the 'Gemma Journey' sample game demonstrating prompt-based NPC dialogue. It also mentions collaboration with Nazara Technologies and the 'Home Run: Gemini Coach Edition' demo using Gemini 2.0 Flash for real-time game coaching, leveraging Vertex AI, Agones, and GKE for scalable game development.
  8. Making it easier to build with the Gemini API in Google AI Studio- Google Developers BlogThis post announces updates to Google AI Studio, including a refreshed UI, a refined prompting interface, and a dedicated developer dashboard. The key enhancement is the revamped Starter Apps experience, which now features native code editing within Google AI Studio. These starter apps are presented as fully functional samples of Gemini model capabilities that developers can edit, share, and integrate into their projects. Examples provided include 'Video Toys', 'Gemini OS', 'GIF Maker', 'Spatial Understanding', and 'Co-drawing', showcasing multimodal understanding, function calling, media generation, and audio streaming.
  9. Gemini 2.5 Flash and Pro, Live API, and Veo 2 in the Gemini API- Google Developers BlogThis post announces the general availability of Gemini 2.5 Pro and the upcoming release of Gemini 2.5 Flash, both featuring 'thinking' capabilities and a one million token context window. It also announces that Veo 2 is production-ready for video generation (text-to-video and image-to-video) with specific quality, length, and pricing details. The Live API for Gemini models is now in preview, offering enhanced rate limits and features for real-time audio, video, and text processing, including expanded language support, configurable VAD, and virtually infinite sessions.
  10. Experiment with Gemini 2.0 Flash native image generation- Google Developers BlogThis post announces the developer experimentation availability of native image generation in Gemini 2.0 Flash, accessible via Google AI Studio and the Gemini API. It highlights four key use cases: text and image storytelling with consistent characters/settings, multi-turn conversational image editing, world understanding for realistic image generation (e.g., recipes), and improved text rendering for advertisements and invitations. It provides a Python code example using the `genai` library to generate an illustrated story.
  11. CalCam: Transforming Food Tracking with the Gemini API- Google Developers BlogThis post details how Polyverse uses the Gemini API (specifically Gemini 2.0 Flash) to power their food tracking app, CalCam. Key contributions include: 1. Demonstrating improved speed (approx. 1 second faster than previous models) and responsiveness with Gemini 2.0 Flash. 2. Reporting a 20% increase in user satisfaction with food recognition results due to Gemini 2.0 Flash's accuracy. 3. Highlighting the benefit of structured JSON output from Gemini 2.0 Flash for seamless integration of dish names, ingredients, macronutrients, and nutritional ratings. 4. Showcasing how Google AI Studio's structured output visual editor simplified development and allowed non-programmers to contribute. The workflow involves image upload, Gemini Flash recognition/analysis, structured output refinement through a secondary workflow, and presentation of nutritional insights. Polyverse plans to leverage Gemini models for AI-driven recipes and coaching.
  12. Start building with Gemini 2.0 Flash and Flash-Lite- Google Developers BlogIntroduces Gemini 2.0 Flash and 2.0 Flash-Lite, highlighting their improved performance, simplified pricing, and extended context capabilities. Provides examples of their use in voice AI (Daily.co), data analytics (Dawn), and video editing (Mosaic), showcasing benefits like reduced search times, cost savings, and faster editing workflows.
  13. Gemini 2.0: Flash, Flash-Lite and Pro- Google Developers BlogIntroduces Gemini 2.0 Flash (GA with higher rate limits, stronger performance, simplified pricing, 1M token context, native tool use, multimodal input), Gemini 2.0 Flash-Lite (public preview, cost-efficient), and Gemini 2.0 Pro (experimental update for coding/complex prompts). Details performance improvements over Gemini 1.5 and pricing updates for Flash and Flash-Lite.
  14. Dive Deep into Gemini: Explore Starter Apps in AI Studio- Google Developers BlogThis post introduces three new starter applications for Gemini 2.0: Spatial Understanding (demonstrating 2D/3D bounding box capabilities for scene analysis), Video Analyzer (a framework for video summarization, object detection, and tracking), and Map Explorer (integrating Gemini with Google Maps API for location-based discovery and interactions). All apps are open-sourced on GitHub and integrated with Google AI Studio, providing developers with ready-to-use codebases for rapid prototyping and customization of AI-powered applications.
  15. Introducing the Android XR SDK Developer Preview- Google Developers BlogIntroduces the Android XR SDK developer preview, a new platform for extended reality (XR) headsets built on existing Android foundations. Highlights include spatialization of apps, automatic availability of most Android apps on XR, and specific development paths for Kotlin/Android Studio (Jetpack XR SDK), Unity (Android XR Extensions), web (WebXR), and native languages (OpenXR 1.1). Details Jetpack XR SDK components: Compose for XR, Material Design for XR, Jetpack SceneCore, and ARCore for Jetpack XR. Mentions new Android Studio Meerkat tools: XR Emulator, XR template, and Layout Inspector. Describes Unity integration with Unity 6 and specific XR packages. Emphasizes WebXR support and open standards like OpenXR 1.1 with vendor extensions (hand mesh, depth textures, light estimation, trackables), glTF 2.0, and OpenEXR.
  16. The next chapter of the Gemini era for developers- Google Developers BlogThis post announces Gemini 2.0 Flash, highlighting its improved performance (twice as fast as 1.5 Pro), new multimodal outputs (text, audio, image generation with SynthID watermarking), native tool use (Google Search, code execution, function calling), and the Multimodal Live API for real-time audio/video streaming. It also introduces experimental AI coding agents like Jules, which can handle bug fixes and code tasks asynchronously within GitHub workflows, and enhancements to Colab's Data Science Agent for notebook generation. The post details early access programs and upcoming integrations into developer platforms.
  17. Looking back at the first year of the Gemini era- Google Developers BlogThis post celebrates the first year of Gemini and Gemma models. It highlights the introduction of Gemini 1.5 Flash and Flash-Lite for faster, cost-effective applications, and Gemini Nano for on-device capabilities. It also details the release of Gemma, a family of open models, with new versions like Gemma 2 (2B, 9B, 27B parameters) and research models like DataGemma, GemmaScope, and PaliGemma 2. The post emphasizes the integration of Gemini into various developer tools (Android Studio, Chrome Dev Tools, Colab, Firebase, Google Cloud, IDX, Gemini Code Assist) for enhanced productivity and mentions experimental data science agents.
  18. 7 examples of Gemini’s multimodal capabilities in action- Google Developers BlogThis post demonstrates Gemini's multimodal capabilities by showcasing seven real-world examples of its image and video understanding. It details how Gemini 1.5 Pro can be used for detailed image descriptions, processing long PDF documents (including extracting tables, charts, and generating code for visualization), reasoning over receipts to extract user-defined fields into JSON, and extracting structured data from webpages. The examples include specific prompts and responses, illustrating the model's ability to handle complex visual and textual information.
  19. OpusClip achieves 30% cost savings in visual description processing with Gemini Flash- Google Developers BlogOpusClip integrated Gemini 1.5 Flash for its 'ClipAnything' feature, using its multimodal capabilities to analyze visuals, actions, emotions, audio, and dialogue for detailed visual descriptions. This resulted in a 30% cost saving in visual description processing, a 30% increase in user engagement for prompt-based features, and a 10% increase in export rates. The Gemini API SDK and support streamlined development and enabled faster time-to-market.