TechPrufer
Get started
Google
Google logoGoogle · Feature trail

AI-Assisted Spec-Driven Development

30 posts · Jan 2024 – Sep 2026

What this trail covers

The AI-Assisted Spec-Driven Development feature thread continues to evolve with advancements in multimodal AI models like Gemini. This includes enabling the analysis of complex, non-RGB data like multi-spectral imagery by mapping invisible bands to visible color channels. This allows for direct application of powerful reasoning engines on specialized datasets without requiring custom model training, significantly lowering the barrier to entry for fields like remote sensing, precision agriculture. This thread now also includes detailed microbenchmarking suites for evaluating accelerator performance, such as TPUs, across network, compute, memory, and host transfer metrics. These benchmarks provide granular insights into hardware capabilities, identify bottlenecks using the Roofline model, and guide performance tuning for large-scale AI workloads, including specific optimizations for transformer-based models and hardware-aware architecture co-design.

All Google trails
  1. The Anatomy of Harness Engineering: How to Evaluate, Iterate, and Guard AI Coding Agents- Google Developers BlogThis post introduces the concept of 'harness engineering' for AI coding agents, focusing on behavioral evaluations as a more reliable measure of confidence than end-to-end benchmarks. It details a paradigm shift from 'report cards' to 'behavioral guideposts', emphasizing discrete, observable actions rather than composite scores. The post outlines a three-step behavioral testing loop: picking one failure mode, writing flexible assertions based on task complexity (strict single-turn for simple tasks, LLM-as-a-judge for complex ones), and automating batch evaluations for stability monitoring. It provides a Python code example using the Antigravity SDK for a behavioral eval and a shell command for running local tests.
  2. Agent and Model Evaluations in Gemini Enterprise Agent Platform are now GA- Google Developers BlogIntroduces the general availability of agent and model evaluations within the Gemini Enterprise Agent Platform. This feature provides a unified engine for measuring and comparing agents and models using consistent metrics across development and production. Key components include over 20 pre-built metrics (quality, safety, grounding, tool use, reference-based scoring), adaptive rubrics that tailor judging criteria, and the ability to define custom code-based or LLM-as-a-judge metrics. Experiment management allows for local or server-side runs with auditable and reproducible artifacts stored in Cloud Storage. Online monitors integrate with existing telemetry to grade live production traffic, producing score-over-time charts and drift alerts. Case generation and simulation tools (user simulator, environment simulator) are provided to bootstrap evaluation datasets and test agent behavior under various conditions, including simulated backend failures.
  3. Enable on-demand expertise with Agent Skills in Genkit Go- Google Developers BlogIntroduced Agent Skills for Genkit, a mechanism for packaging specialized expertise into discoverable capabilities that agents load only when needed. This leverages a progressive disclosure model where only skill metadata is initially exposed to the agent's system prompt, with the full skill body and bundled resources (scripts, references) loaded dynamically when required. This improves token efficiency, simplifies implementation, and allows for resource bundling. Demonstrated integration with Genkit Go via middleware hooks, including discovery, activation, and execution stages, with code examples for a recipe generation tool and a multi-modal art restoration application.
  4. How to use Google microbenchmarks for evaluating TPU performance- Google Developers BlogIntroduces a microbenchmark suite for evaluating TPU performance across network, compute, HBM, and host transfer categories. Details key metrics like TFLOPs, MFU, and memory bandwidth. Explains how microbenchmarks inform the Roofline model to diagnose compute, memory, or network-bound bottlenecks. Presents a case study on optimizing a 110B MoE training workload on TPU 7x, highlighting the impact of microbenchmark-driven interventions like SparseCore collective offloading and Tokamax Splash Attention.
  5. Building scalable AI agents with modular prompt transpilation- Google Developers BlogThis post introduces a production prompt transpiler that treats AI agent prompts as build artifacts rather than static text. It advocates for modular skill files, dependency management, build-time validation (e.g., for missing imports, undefined variables, circular dependencies), and drift checking through CI pipelines. The system supports dynamic skills and agent-authored updates, where agents can propose changes to their instruction layers via pull requests, which are then subjected to validation and review. This approach aims to improve prompt maintainability, reduce blast radius, prevent copy-paste drift, and defer runtime errors, ultimately enhancing agent reliability.
  7. Evolving Spec-Driven Development: Conductor Now Supports Antigravity- Google Developers BlogThis post announces the evolution of Conductor from a Gemini CLI extension to a portable plugin. This transition enables a more conversational interaction model for spec-driven development (SDD), where AI agents dynamically generate context, specs, and plans through natural conversation. The plugin architecture ensures compatibility with various tools like Antigravity CLI, Claude, and others, allowing for seamless continuation of AI-assisted workflows across different environments. The post highlights improved task completion rates compared to non-SDD workflows and emphasizes backward compatibility with existing plans and specs.
  8. LiteRT.js, Google's high performance Web AI Inference- Google Developers BlogThis post introduces LiteRT.js, a JavaScript binding for the LiteRT on-device inference library, enabling high-performance AI model execution directly in web browsers via WebAssembly. It highlights the use of hardware acceleration through XNNPACK (CPU), ML Drift (GPU via WebGPU), and WebNN (NPU). It details PyTorch model conversion with LiteRT Torch and tailored quantization with AI Edge Quantizer. Performance benchmarks show up to 3x speedups over other web runtimes and 5-60x speedups for GPU/NPU over CPU. It also showcases integrations with Ultralytics YOLO, Depth Estimation, and Image Upscaling demos.
  9. Bridging the Domain Gap: AI Race Coach built with Antigravity and Gemini- Google Developers BlogThis post details the development of an AI Race Coach using Antigravity and Gemini, showcasing real-time telemetry ingestion, edge reasoning with Gemma 4, and cloud reasoning with Gemini API. It highlights the use of Antigravity for domain bridging, Python for data parsing, Jetpack Compose for the UI, and custom hardware for direct vehicle telemetry capture. A key technical achievement was the activation of the Pixel 10 TPU, enabling 40 tokens per second processing for real-time coaching alerts.
  10. Build agentic full-stack apps with Genkit- Google Developers BlogThis post announces Genkit Agents, an open-source framework for building full-stack, AI-powered and agentic applications. It introduces the Agents API for defining and driving agents with a unified chat interface, supporting features like message history, tool loops, streaming, persistence, and frontend protocols. Agents can be server-managed or client-managed, with options for storing custom state and artifacts. Genkit supports HTTP serving and offers a JavaScript client for full-stack integration. It also includes features for human approval in agent workflows and for handling long-running tasks. The post also clarifies when to use ADK instead of Genkit.
  11. ML Development in VS Code with Google Cloud Power: Workbench Extension Now Available- Google Developers BlogIntroduces the Google Cloud Workbench Notebooks extension for VS Code, enabling users to connect their local IDE to managed cloud environments for ML development. This extension streamlines the ML lifecycle by eliminating context switching and allowing users to run notebooks directly on Workbench instances. It also highlights the open-sourced nature of the extension and provides steps for installation and connection.
  12. Why we built ADK 2.0- Google Developers BlogThis post introduces ADK 2.0, which enhances AI agent development by providing a structured workflow runtime and task-collaboration model. It allows for the seamless blending of deterministic execution logic with LLM-based agents, addressing issues like context bloat, execution derailment, and prompt injection. The post details the benefits of this approach through a customer refund processing example, highlighting efficiency gains in token usage and latency, and introduces dynamic workflows for complex business logic and structured multi-agent collaboration.
  13. Build reliable multi-agent applications with ADK Go 2.0. Discover our new graph-based workflow engine, built-in human-in-the-loop, and dynamic orchestration- Google Developers BlogADK for Go 2.0 introduces a graph-based workflow engine for composing multi-agent applications, built-in human-in-the-loop (HITL) as a primitive, dynamic orchestration in Go, LLM agent modes, and a unified node runtime. It enhances resilience with built-in retry policies, timeouts, and concurrency controls. The update also unifies agent and workflow execution models and introduces a single `agent.Context` for tools, callbacks, and workflow nodes.
  14. Driving the Agent Quality Flywheel from Your Coding Agent- Google Developers BlogThis post introduces a "quality flywheel" for engineering AI agents, driven by a coding agent that automates the evaluation and refinement cycle. It details a five-stage process: Prepare Data, Run Inference, Grade, Analyze Failures, and Optimize & Iterate. Key components include adaptive AutoRaters for grading, Automatic Loss Analysis for clustering failures, and a decoupled optimizer/evaluator architecture. The post also highlights the development of custom metrics to isolate specific failure modes, such as mid-conversation changes in agent behavior, and provides concrete examples of how the system operates using the `_travel-concierge_` agent.
  15. Build Cross-Language Multi-Agent Team with Google’s Agent Development Kit and A2A- Google Developers BlogThis post introduces the practical application of the Agent Development Kit (ADK) and the Agent-to-Agent (A2A) protocol for building cross-language multi-agent systems. It details how to create a contract compliance pipeline with a Python agent (using Gemini for extraction) and a Go agent (for deterministic validation), connected via A2A. Key contributions include demonstrating the A2A protocol's discovery (Agent Cards) and communication (JSON-RPC 2.0 with Message Parts) mechanisms, the use of ADK's RemoteA2aAgent for abstracting remote A2A services, and the concept of shared state (ToolContext.state) for inter-agent communication. It also highlights production-ready patterns like state management for fault tolerance (MANUAL_REVIEW) and the decomposition of monolithic agents into specialized microservices.
  16. Measuring What Matters with Jules- Google Developers BlogIntroduces a new methodology for evaluating proactive AI coding agents by leveraging historical bug-fixing data to establish 'ground truth' for 'aspirational goals'. Details a preliminary benchmark using 705 bugs from internal Google codebases to cluster related bugs, set them as targets, and test an agent's ability to identify insights within an exploration budget. Presents preliminary results showing the agent's effectiveness in identifying relevant insights and the benefit of increased exploration budgets. Outlines plans to expand the evaluation to public GitHub data and ingest richer context streams.
  17. How A2A is Building a World of Collaborative Agents- Google Developers BlogThis post introduces the Agent-to-Agent (A2A) protocol, a new communication standard designed for collaborative AI agents. It details the architectural advantages of A2A, including secure boundaries for sensitive data, zero context pollution by offloading dependencies to peer agents, dynamic autonomy enabling agents to refine plans and ask clarifying questions, and workload distribution for modular development. The post highlights the FoldRun agent as a practical application in life sciences for protein structure prediction, demonstrating how A2A simplifies complex workflows. It also outlines potential applications in agentic commerce, enterprise data streaming, cross-platform IT/DevOps, and secure telecommunications. The post also announces the availability of A2A SDKs in Python and Go (GA), Java (Beta), .NET (Preview), and JavaScript/TypeScript.
  18. Announcing ADK for Kotlin and ADK for Android 0.1.0: Building AI Agents on Android and Beyond- Google Developers BlogAnnounces ADK for Kotlin 0.1.0 and ADK for Android 0.1.0. ADK for Kotlin enables agentic workflows on backend projects, supporting hybrid orchestration (cloud orchestrator with on-device sub-agents), on-device sequential agents, local retrieval, and flexible tooling. ADK for Android provides specialized on-device optimizations for AI agents within Android apps, leveraging local LLMs (like Gemini Nano) for enhanced privacy and the ability to bridge with cloud models. The post provides code examples for building agents in Kotlin and integrating ADK into Android projects, highlighting features like model swapping, session state sharing, and tool integration using annotations.
  19. An important update: Transitioning Gemini CLI to Antigravity CLI- Google Developers BlogThis post announces the transition of Gemini CLI to Antigravity CLI, a new terminal experience within the Google Antigravity agent-first development platform. Key features from Gemini CLI such as Agent Skills, Hooks, Subagents, and Extensions (now plugins) are carried over. Antigravity CLI is built in Go for improved performance and supports asynchronous workflows for multi-agent orchestration. It shares a unified architecture with Antigravity 2.0. The post outlines a timeline for consumers and enterprise customers regarding the deprecation of Gemini CLI and IDE extensions, and the availability of Antigravity CLI and Antigravity 2.0.
  20. Blazing fast on-device GenAI with LiteRT-LM- Google Developers BlogThis post introduces LiteRT-LM, a new runtime for on-device GenAI, specifically optimized for Gemma 4 models. It details performance improvements achieved through advanced quantization, XNNPACK/MLDrift kernels, and optimized data transfer avoidance. Key contributions include the implementation of Multi-Token Prediction (MTP) for a 2.2x speedup by executing drafter and primary models on the same hardware IP, advanced session management for KV cache serialization, and memory footprint optimizations for multimodal capabilities. It also highlights Thinking Mode for reasoning and constrained decoding for structured output, along with native function-calling support. New Swift and JavaScript APIs are introduced for iOS and web (WebGPU) respectively.
  21. Google Tensor SDK Beta with LiteRT- Google Developers BlogThis post announces the Beta release of the Google Tensor ML SDK, which integrates with LiteRT for on-device ML deployment on Pixel devices. It details the unified developer workflow provided by LiteRT, including model compilation (PyTorch/TFLite to optimized binaries), deployment via Play Feature Delivery and AI Packs, and running inference with LiteRT Runtime. It also highlights the Model Garden with over 100 models and precompiled options on Hugging Face. The post emphasizes the ability to run Small Language Models (like Function Gemma and Embedding Gemma), intelligent content creation, vision/understanding, and audio/accessibility features on-device.
  22. Announcing Genkit Middleware: Intercept, extend, and harden your agentic apps- Google Developers BlogThis post introduces the Genkit middleware system, a new capability for intercepting and extending generation calls in agentic applications. It details the three layers of middleware hooks (Generate, Model, Tool) and provides examples of pre-built middleware for retry, fallback, tool approval, skills injection, and filesystem access. The post also demonstrates how to build custom middleware, such as a content filter, and explains how middleware stacks are composed. The Developer UI's integration with middleware execution is also highlighted.
  23. Build Long-running AI agents that pause, resume, and never lose context with ADK- Google Developers BlogThis post introduces the concept of building long-running AI agents using the Agent Development Kit (ADK) that can pause, resume, and maintain context. It details three key architectural shifts: durable memory schemas, event-driven dormancy gates, and multi-agent delegation. The post demonstrates how to implement these concepts using a state machine for tracking onboarding progress, wiring the state into the agent's system instruction, using tools to advance the state machine, implementing checkpoint-and-resume with persistent sessions (SQLite/Cloud SQL), and handling idle time with event-driven resumption via webhook endpoints. It also highlights the use of a coding agent and Agents CLI for scaffolding and iterative development.
  24. Building with Gemini Embedding 2: Agentic multimodal RAG and beyond- Google Developers BlogThis post introduces Gemini Embedding 2, a multimodal embedding model that maps text, images, video, audio, and documents into a single embedding space. It details its application in agentic multimodal RAG using task prefixes for optimized embeddings, multimodal search with specific prefixes, search reranking via similarity metrics, and clustering, classification, and anomaly detection using symmetric task prefixes. The post also highlights efficient storage options with Matryoshka Representation Learning (MRL) and cost-saving batch processing.
  25. Building real-world on-device AI with LiteRT and NPU- Google Developers BlogThis post introduces LiteRT, a cross-platform framework for on-device AI that enables CPU, GPU, and NPU acceleration. It abstracts vendor-specific NPU SDK complexities, allowing developers to target diverse silicon with a unified API. Real-world examples from Google Meet, Epic Games, and Argmax Inc. demonstrate significant performance gains (e.g., 25x larger models in Google Meet, 2x speedup from GPU to NPU in Argmax Pro SDK) and power efficiency improvements. LiteRT extends NPU acceleration beyond mobile to industrial IoT and AI PCs through integrations with platforms like Qualcomm Dragonwing™ IQ8 Series and Intel® Core™ Ultra processors via OpenVINO™. The Google AI Edge Portal provides benchmarking services for performance validation. LiteRT supports both JIT and AOT deployment, with open models available on Hugging Face and experimental access to the Google Tensor ML SDK.
  26. Build Better AI Agents: 5 Developer Tips from the Agent Bake-Off- Google Developers BlogThis post details 5 key developer tips for building production-grade AI agents, derived from the Google Cloud AI Agent Bake-Off. It emphasizes moving beyond simple prompt engineering to rigorous 'agentic engineering'. Key takeaways include: 1. Architecting for multi-agent workflows by decomposing problems into specialized sub-agents managed by a supervisor agent, similar to microservices. 2. Building agent harnesses with a mindset of impermanence, expecting them to be replaced by advancements in state-of-the-art models. 3. Treating multimodality as a core requirement, not an add-on, by natively integrating multimodal models for richer context ingestion and generation. 4. Leveraging open source protocols like MCP, A2A, UCP, AP2, A2UI, and AG-UI for dynamic resource discovery and communication, reducing the need for custom integration code. 5. Using deterministic code (e.g., Python functions, SQL queries) for execution after LLMs perform reasoning and intent extraction, with strict JSON validation schemas to capture and validate model outputs.
  27. Introducing Finish Changes and Outlines, now available in Gemini Code Assist extensions on IntelliJ and VS Code- Google Developers BlogIntroduced 'Finish Changes' and 'Outlines' features to Gemini Code Assist extensions for IntelliJ and VS Code. 'Finish Changes' allows AI to complete code based on in-progress modifications, pseudocode, comments, and demonstrated patterns, reducing the need for explicit prompts. 'Outlines' generates inline code summaries for improved comprehension and navigation within the IDE. Both features are powered by Gemini 3.0 and aim to streamline developer workflows and reduce ramp-up time.
  28. Unleash Your Development Superpowers: Refining the Core Coding Experience- Google Developers BlogIntroduced Agent Mode with Auto Approve for automated multi-file code changes, Inline Diff Views for direct editing of AI-generated code, Revert to Checkpoint for rolling back changes, Multi-part Chat Code Suggestions for granular acceptance of AI code, Chat Code Suggestion Previews for scannable code blocks, and improved code completion speed. Enhanced user control with Context Drawer for managing conversational context, Custom Commands for workflow automation, adding code snippets and terminal output to chat context, configurable codebase awareness via AI exclusion files, and a stop button for in-progress chat responses. Added 'thinking tokens' and release channel name in chat banner for transparency.
  29. Conductor Update: Introducing Automated Reviews- Google Developers BlogIntroduced Automated Review feature for Conductor, extending its capabilities from planning and execution to validation. This feature generates post-implementation reports covering code review (static/logic analysis), plan compliance, guideline enforcement, test-suite validation, and basic security review. The output is categorized by severity (High, Medium, Low) with actionable insights for developers.
  30. LiteRT: The Universal Framework for On-Device AI- Google Developers BlogThis post announces the full production release of LiteRT as the universal on-device inference framework for the AI era, building upon its TensorFlow Lite foundation. Key advancements include 1.4x faster GPU performance than TFLite, new NPU acceleration, a unified workflow for GPU and NPU acceleration across Android, iOS, macOS, Windows, Linux, and Web, superior cross-platform GenAI deployment for models like Gemma, and first-class PyTorch/JAX support via seamless model conversion. It highlights comprehensive GPU support via OpenCL, OpenGL, Metal, and WebGPU through ML Drift, achieving average 1.4x faster performance than TFLite GPU delegate, with optimizations like asynchronous execution and zero-copy buffer interoperability yielding up to 2x faster performance. LiteRT simplifies NPU integration with a three-step process and offers AOT and JIT compilation. Integrations with MediaTek and Qualcomm NPUs achieve significant speed improvements. The post details LiteRT's support for popular open-weight models and its integrated tech stack for GenAI deployment, including LiteRT Torch Generative API, LiteRT-LM, and LiteRT Converter & Runtime.
  31. Conductor: Introducing context-driven development for Gemini CLI- Google Developers BlogIntroduces Conductor, a Gemini CLI extension that enables context-driven development by managing project context (product, tech stack, workflow) in persistent Markdown files. This allows for formal specs and plans to guide AI agents, supporting brownfield projects and team collaboration by ensuring AI adheres to established guidelines and best practices. It details the workflow for establishing context, specifying and planning tracks (specs and plans), and implementing code based on these artifacts.
  32. MediaTek NPU and LiteRT: Powering the next generation of on-device AI- Google Developers BlogThis post introduces the LiteRT NeuroPilot Accelerator, a successor to TFLite NeuroPilot delegate, enabling seamless deployment of AI models on MediaTek NPUs. It highlights a unified development workflow with AOT and on-device compilation options, simplified C++ API with Native Hardware Buffer Interoperability for zero-copy data passing, and rich generative AI capabilities with optimized support for models like Gemma and Qwen. Performance benchmarks show significant acceleration for Gemma models on MediaTek chipsets compared to CPU and GPU. The post also details integration paths for developers using LiteRT-LM for text generation and LiteRT for embedding models.
  33. Unlocking Peak Performance on Qualcomm NPU with LiteRT- Google Developers BlogIntroduces the LiteRT Qualcomm AI Engine Direct (QNN) Accelerator, a successor to the TFLite QNN delegate, for on-device ML inference on Qualcomm NPUs. This new accelerator simplifies mobile deployment by abstracting vendor-specific SDKs and SoC fragmentation, offering a unified workflow for AOT or on-device compilation. It achieves state-of-the-art performance for LLMs and GenAI models, with benchmarks showing up to 100x speedup over CPU and 10x over GPU on Qualcomm's latest SoCs. The post details performance gains for models like FastVLM-0.5B, including low time-to-first-token and high token throughput, and provides a 3-step guide for developers to integrate LiteRT QNN acceleration into their Android applications, including AOT compilation with AI Pack export and deployment via Google Play for On-device AI.
  34. Build with Google Antigravity, our new agentic development platform- Google Developers BlogIntroduces Google Antigravity, a new agentic development platform that combines an AI-powered coding experience with a dedicated agent-first interface (Manager Surface). It enables autonomous planning, execution, and verification of complex tasks across editor, terminal, and browser. Key features include an Editor View for synchronous workflows and a Manager Surface for asynchronous agent orchestration. Practical use cases demonstrate delegating multi-tool software tasks, requesting UI changes with artifact-based verification, and background maintenance/bug fixing. Artifacts (task lists, plans, screenshots, recordings) are used for verification, with direct feedback integration. The platform supports learning through context saving and offers model optionality with Gemini 3 Pro, Claude Sonnet 4.5, and GPT-OSS.
  35. Building with Gemini 3 in Jules- Google Developers BlogThis post announces the integration of Gemini 3 Pro into Jules, an autonomous coding agent. Gemini 3 Pro enhances Jules' agentic capabilities, leading to clearer reasoning, stronger intent alignment, and improved reliability for multi-step tasks. Jules is now available on more surfaces (terminal, Gemini CLI extension, API) with a unified project view. Recent improvements to Jules include parallel CLI runs, Windows compatibility, a simpler invocation flow, a diff viewer, a stable API for custom workflows, support for environment variables, improved session memory, more reliable critic agent replanning, safer Git handling, and improved VM performance. Future work includes direct directory attachment, automatic title updates, web UI CLI shortcuts, simplified start modes, and automatic PR creation.
  36. Introducing Code Wiki: Accelerating your code understanding- Google Developers BlogIntroduces Code Wiki, a platform that generates and maintains continuously updated, structured wikis for code repositories. It integrates with a Gemini-powered chat agent that uses the wiki as context to answer repository-specific questions. The system automatically regenerates documentation after code changes and links answers directly to relevant code files, classes, and functions. It also generates architecture, class, and sequence diagrams. A Gemini CLI extension for Code Wiki is announced for internal repositories.
  37. Introducing the Jules extension for Gemini CLI- Google Developers BlogIntroduces the Jules extension for Gemini CLI, enabling asynchronous task delegation to an autonomous sidekick. Jules operates in a VM, clones code, installs dependencies, and modifies files. It supports asynchronous tasks, background bug fixing, and submitting changes to new branches. The post details prerequisites (Gemini CLI, Jules Account, repository connection) and installation/usage instructions for the Jules extension.
  38. Announcing Genkit Go 1.0 and Enhanced AI-Assisted Development- Google Developers BlogAnnounces Genkit Go 1.0, the first stable release of the AI development framework for Go. Introduces `genkit init:ai-tools` command for integrating AI coding assistants. Highlights type-safe AI flows using Go structs, a unified model interface supporting various providers (Google AI, Vertex AI, OpenAI, Ollama), tool calling capabilities, and easy deployment options. Details rich developer tooling including a standalone CLI and interactive Developer UI. Provides a quick start guide for new projects.
  39. Announcing the Genkit Extension for Gemini CLI- Google Developers BlogIntroduces the Genkit Extension for Gemini CLI, which provides deep integration between Gemini CLI and Genkit projects. This extension bundles Genkit's MCP server with specialized context files, giving Gemini CLI a comprehensive understanding of the Genkit SDKs. Key features include Genkit-specific MCP tools (`get_usage_guide`, `lookup_genkit_docs`, `list_flows`, `run_flow`, `get_trace`) for context-aware assistance and intelligent code generation tailored for Genkit application development. It also emphasizes seamless workflow integration with the Genkit Developer UI and supports Google Gen AI providers.
  40. Own your AI: Learn how to fine-tune Gemma 3 270M and run it on-device- Google Developers BlogThis post details the process of fine-tuning Gemma 3 270M for a specific task (text-to-emoji translation) using QLoRA for parameter-efficient fine-tuning. It then describes quantizing the model to under 300MB for on-device deployment and running it client-side in a web app using MediaPipe or Transformers.js, leveraging WebGPU for local computation. This demonstrates a practical workflow for creating and deploying specialized, on-device AI models.
  41. Level Up Your Dev Game: The Jules API is Here!- Google Developers BlogIntroduces the Jules API, a new system for automating and integrating across the software development lifecycle. Details the core concepts of the Jules API: Source (e.g., GitHub repository), Session (an asynchronous coding agent's continuous block of work), and Activity (individual steps within a session). Provides examples of use cases such as fixing bugs from Slack and automating backlog triage. Includes a quickstart guide with `curl` commands to get an API key, list sources, and initiate a session.
  42. Gemini 2.5 Flash Image now ready for production with new aspect ratios- Google Developers BlogThis post announces the general availability of Gemini 2.5 Flash Image for production environments. Key new features include support for 10 different aspect ratios (landscape, square, portrait, and flexible) and the ability to specify image-only output. It highlights use cases from Cartwheel and Volley, demonstrating character consistency and in-session editing. The post also provides links to developer documentation, a cookbook, and example apps in Google AI Studio, along with pricing details and a Python code sample.
  43. Meet Jules Tools: A Command Line Companion for Google’s Async Coding Agent- Google Developers BlogThis post introduces Jules Tools, a command-line interface (CLI) and terminal user interface (TUI) for interacting with the Jules asynchronous coding agent. It details installation via npm, command and flag usage, and provides practical examples of integrating Jules Tools with other CLI tools (e.g., `gh`, `jq`, `gemini`) for tasks like listing repositories, creating remote sessions, assigning sessions from TODO files, and piping GitHub issues to Jules. The TUI offers a dashboard view and step-by-step creation of tasks. The post also outlines the hybrid development philosophy behind Jules Tools, combining local and remote execution with DIY and delegation approaches.
  44. Unlocking Multi-Spectral Data with Gemini- Google Developers BlogThis post introduces a technique for enabling Gemini models to analyze multi-spectral imagery by mapping invisible spectral bands (NIR, SWIR) to RGB channels, creating 'false-color composite' images. This process involves selecting relevant bands, normalizing their data to a 0-255 range, and assigning them to R, G, and B channels. The key innovation is using detailed prompts to teach the model how to interpret these custom images in real-time, allowing it to perform tasks like vegetation health assessment, water detection, and burn scar identification without requiring specialized model retraining. The post provides examples and a Colab notebook demonstrating this capability with the EuroSat dataset and highlights its potential for rapid prototyping in remote sensing applications.
  45. Gemma explained: EmbeddingGemma Architecture and Recipe- Google Developers BlogThis post introduces EmbeddingGemma, a new text embedding model derived from Gemma 3. It details the model's architecture, which is an encoder-decoder derived from a decoder-only Gemma 3 model using T5Gemma adaptation. The post explains the embedding generation process, including Transformer, Pooling (Mean Pooling), Dense layers, and Normalize steps. It elaborates on the three loss functions used for training: Noise-Contrastive Estimation (NCE) for similarity/contrast, Global Orthogonal Regularizer (GOR) for embedding space distribution, and Geometric Embedding Distillation from a larger Gemini model. Matryoshka Representation Learning (MRL) is introduced as a technique for variable embedding dimensionality. The development recipe includes encoder-decoder training, pre fine-tuning, fine-tuning with hard negatives, model soup averaging, and Quantization-Aware Training (QAT).
  46. Continuing to bring you our latest models, with an improved Gemini 2.5 Flash and Flash-Lite release- Google Developers BlogThis post announces updated versions of Gemini 2.5 Flash and 2.5 Flash-Lite. Gemini 2.5 Flash-Lite has improved instruction following, reduced verbosity (leading to 50% output token reduction), and stronger multimodal/translation capabilities. Gemini 2.5 Flash has improved agentic tool use (5% gain on SWE-Bench Verified) and is more cost-efficient with thinking on (fewer tokens, reduced latency). New `-latest` aliases (`gemini-flash-latest`, `gemini-flash-lite-latest`) are introduced for easier access to the most recent preview models.
  47. Gemini CLI 🤝 FastMCP: Simplifying MCP server development- Google Developers BlogThis post details the integration of FastMCP with Gemini CLI, enabling the installation of local STDIO transport MCP servers using the `fastmcp install gemini-cli server.py` command. This integration simplifies the configuration and makes FastMCP server capabilities available within Gemini CLI's MCP management system. It highlights how MCP servers, built with FastMCP's Pythonic design, can enhance Gemini CLI by providing tools for actions and context access, and prompts that can be used as slash commands.
  48. Veo 3 and Veo 3 Fast – new pricing, new configurations and better resolution- Google Developers BlogIntroduced new pricing for Veo 3 and Veo 3 Fast, making them more affordable. Added support for vertical format outputs (9:16 aspect ratio) and 1080p HD resolution for Veo 3 and Veo 3 Fast. Made Veo 3 and Veo 3 Fast stable and production-ready in the Gemini API. Provided a Python code example demonstrating how to generate videos using Veo 3 Fast, including setting negative prompts, aspect ratio, and resolution. Highlighted integrations with MediaSim demo app, tldraw SDK, Invisible Studio, Saga, and Mosaic.
  49. Beyond the terminal: Gemini CLI comes to Zed- Google Developers BlogThis post announces the integration of Gemini CLI with the Zed code editor. It highlights how this integration brings AI capabilities directly into the editor, enabling features like in-place code generation/refactoring, instant code explanations, and natural language chat within the terminal. The collaboration emphasizes a new workflow for interacting with AI agents, including real-time progress following, pull-request-style code review, seamless context switching, and the ability to provide external context (URLs) to the agent. The post also mentions the extensibility of Gemini CLI through MCP, markdown-based instructions, custom extensions, and tools.
  50. What's new in Gemini Code Assist- Google Developers BlogThis post announces the general availability of Gemini Code Assist's Agent Mode in VS Code (Preview) and IntelliJ (Stable). It details significant improvements to the Agent Mode experience, including inline diffing, persistent chat history, real-time shell command output, and batched tool call approvals in VS Code. For IntelliJ, it highlights thinking tokens for better insight into Gemini's thought process and the option for auto-approve mode. The post also covers IDE improvements like the ability to stop in-progress chat responses and generally available preview features. Furthermore, it details updates to the Gemini CLI, including the launch of Gemini CLI GitHub Actions (Beta) for automated repository tasks and deeper VS Code integration with context-awareness and native diffing. Custom slash commands and a public roadmap for Gemini CLI v1 are also announced.
  51. Gemini CLI + VS Code: Native diffing and context-aware workflows- Google Developers BlogThis post introduces a significant enhancement to the Gemini CLI by enabling deep integration with VS Code. This integration provides the CLI with workspace and selection context, allowing it to offer more relevant suggestions. Furthermore, it introduces native in-editor diffing within VS Code for reviewing and modifying suggested code changes before acceptance. New commands `/ide install`, `/ide enable`, and `/ide disable` are introduced to manage this IDE integration.
  52. Meet Jules’ sharpest critic and most valuable ally- Google Developers BlogIntroduces a "critic" functionality integrated into the AI coding agent (Jules) to review and critique generated code before it is presented to the user. This critic-augmented generation process involves an adversarial review of proposed code changes, flagging issues like subtle logic errors, missed edge cases, and inefficient algorithms. The critic operates in a one-shot pass for the initial version, with future plans for multi-step agent capabilities. The goal is to improve code quality, test coverage, and security by moving the review process earlier in the development cycle.
  53. Gemini Embedding: Powering RAG and context engineering- Google Developers BlogThis post details the adoption and performance of the Gemini Embedding text model (`gemini-embedding-001`) across various organizations for Retrieval Augmented Generation (RAG) and context engineering. It highlights specific performance metrics such as increased recall (Box: 3.6%), improved F1 scores (re:cap: 1.9% and 1.45%), higher accuracy in legal discovery (Everlaw: 87%), reduced embedding time (Poke: 90.4% reduction), and consistent sub-second latency (Mindlid: median 420ms). The post also mentions the Matryoshka property of Gemini Embedding for compact representations and its multilingual support.
  54. Introducing Opal: describe, create, and share your AI mini-apps- Google Developers BlogIntroduces Opal, an experimental tool for building and sharing AI mini-apps. Opal allows users to create workflows by chaining prompts, AI model calls, and tools using natural language and a visual editor, without requiring code. It supports sharing these apps for immediate use by others.
  55. Unleashing new AI capabilities for popular frameworks in Firebase Studio- Google Developers BlogIntroduced AI-optimized templates for Flutter, Angular, React, Next.js, and Web within Firebase Studio, enabling autonomous agent mode and redesigned layouts. Added `airules.md` files to templates for specialized Gemini instructions on code generation and documentation. Enhanced integration with Firebase backend services by allowing Gemini to recommend and integrate services via prompts. Introduced the ability to fork workspaces for experimentation, collaboration, and debugging. Increased project upload size to 100MB and added an "enhance prompts" feature for prompt refinement with Gemini.
  56. Conversational image segmentation with Gemini 2.5- Google Developers BlogThis post introduces conversational image segmentation capabilities powered by Gemini 2.5. It details how the model can segment objects based on relational understanding, ordering, comparative attributes, conditional logic (including negations), abstract concepts, in-image text (leveraging OCR), and multi-lingual labels. The post highlights use cases in interactive media editing, intelligent safety and compliance monitoring, and nuanced insurance damage assessment. It emphasizes the flexible language and simplified developer experience offered by a single API for sophisticated vision applications. Best practices for using the gemini-2.5-flash model with specific prompt configurations for JSON output are provided. The post also links to interactive demos and Colab notebooks for developers to start building.
  57. Simplify your Agent "vibe building" flow with ADK and Gemini CLI- Google Developers BlogThis post introduces an enhanced workflow for agent development by combining the Agent Development Kit (ADK) with the Gemini CLI. The core technical contribution is the optimization of the `llms-full.txt` file within the ADK, reducing its size and improving its understandability for LLMs. This allows the Gemini CLI to act as a specialized ADK assistant, enabling conversational generation and modification of ADK-based Python code for AI agents. The post demonstrates this by walking through the creation of a GitHub issue labeling agent, from initial planning via Gemini CLI to code generation and iterative refinement, highlighting reduced friction and accelerated prototyping.
  58. Gemini Embedding now generally available in the Gemini API- Google Developers BlogAnnounces the general availability of the `gemini-embedding-001` model in the Gemini API and Vertex AI. Highlights its performance on the MTEB Multilingual leaderboard, its support for over 100 languages, a 2048 token limit, and the use of Matryoshka Representation Learning (MRL) for adjustable output dimensions. Details pricing at $0.15 per 1M input tokens and provides Python code examples for using the `embed_content` endpoint. Includes deprecation timelines for older embedding models.
  59. Advancing agentic AI development with Firebase Studio- Google Developers BlogIntroduces three new Agent modes (Ask, Agent, Agent Auto-run) for Gemini within Firebase Studio, enabling conversational, supervised, and autonomous AI assistance for development tasks. Adds foundational support for the Model Context Protocol (MCP), allowing integration of MCP servers for extended workflows, with examples of using it to query Cloud Firestore and retrieve context from libraries like MediaPipe. Integrates Gemini CLI directly into Firebase Studio, providing an AI-powered terminal experience for code generation, debugging, and project management. Mentions project-level rule files (.idx/airules.md, GEMINI.md, .cursorrules) for guiding Gemini's behavior.
  60. Supercharge your notebooks: The new AI-first Google Colab is now available to everyone- Google Developers BlogThis post announces the general availability of the AI-first Google Colab experience, which integrates agentic capabilities for accelerating machine learning projects, debugging, and data visualization. It highlights features like iterative querying, a Next-Generation Data Science Agent for autonomous workflows, and effortless code transformation through natural language. The post details how users are leveraging these features for end-to-end ML lifecycle acceleration, smarter debugging with AI pair programming and diff-based fixes, and hassle-free data visualization generation.
  61. Multilingual innovation in LLMs: How open models help unlock global communication- Google Developers BlogThis post details the outcomes of the 'Unlock Global Communication with Gemma' Kaggle competition, showcasing how developers adapted large language models (LLMs), specifically Gemma, for diverse cultural and linguistic contexts. It highlights projects that fine-tuned Gemma for various languages (Swahili, Traditional Chinese, Arabic, Italian, Ancient Chinese, Japanese, Hindi, Kazakh, Old English) and specific tasks (translation, lyric translation, historical text comprehension, mathematical reasoning, Yomigana generation). Techniques discussed include parameter-efficient fine-tuning (PEFT), retrieval augmentation (RA), and post-training methods to mitigate issues like hallucinations and catastrophic forgetting. The post also notes the degradation of conversational capabilities when fine-tuning for singular translation tasks and the potential of LLMs for historical cultural preservation and educational tools.
  62. Gemini 2.5: Updates to our family of thinking models- Google Developers BlogThis post announces the general availability and stability of Gemini 2.5 Pro and Gemini 2.5 Flash, along with the preview release of Gemini 2.5 Flash-Lite. It details pricing updates for Gemini 2.5 Flash and introduces Gemini 2.5 Flash-Lite as a cost-effective, low-latency option for high-throughput tasks. The post highlights the reasoning capabilities of these models and their support for native tools like Grounding with Google Search and Code Execution.
  63. An upgraded dev experience in Google AI Studio- Google Developers BlogIntroduced a native code editor in Google AI Studio powered by Gemini 2.5 Pro for code generation, integrated with the Gen AI SDK. Added a 'Build' tab for creating and deploying AI-powered web apps to Cloud Run with a single click. Enhanced iterative development with chat-based editing, diff views, and version history. Centralized generative media models (Imagen, Veo, Gemini, Lyria RealTime) on a 'Generate Media' page. Added Gemini 2.5 Flash native audio dialog to the Live API with improved naturalness and proactive audio. Introduced Gemini 2.5 Pro and Flash previews for text-to-speech (TTS) with native audio output and flexible control. Added native Model Context Protocol (MCP) support to the Gen AI SDK. Launched an experimental 'URL Context' tool for models to reference external links.
  64. From idea to app: Introducing Stitch, a new way to design UIs- Google Developers BlogIntroduces Stitch, a new experiment from Google Labs that leverages Gemini 2.5 Pro to transform prompt and image inputs into UI designs and frontend code. Stitch offers UI generation from natural language and images/wireframes, rapid iteration capabilities, and seamless integration with development workflows through 'Paste to Figma' and front-end code export.
  65. Fully Reimagined: AI-First Google Colab- Google Developers BlogThis post announces a significant AI-first reimagining of Google Colab, integrating agentic capabilities powered by Gemini 2.5 Flash. Key new features include iterative querying for code generation, transformation, and intelligent error fixing with diff views; an upgraded Data Science Agent (DSA) that can perform autonomous analytical workflows with interactive feedback; and flexible interaction models (center stage box, side panel). The post highlights potential for over 2x efficiency gains based on previous Gemini integrations.
  66. Advancing the frontier of video understanding with Gemini 2.5- Google Developers BlogIntroduces Gemini 2.5 Pro and Gemini 2.5 Flash, highlighting their advanced video understanding capabilities. Demonstrates use cases such as transforming videos into interactive applications (Video To Learning App), generating p5.js animations from video, retrieving specific moments from videos (e.g., Google Cloud Next '25 keynote), and performing temporal reasoning (e.g., counting phone usage in Project Astra video). Mentions availability in Google AI Studio, Gemini API, and Vertex AI, with a new 'low' media resolution parameter for cost-effective long video processing.
  67. Gemini 2.5 Pro Preview: even better coding performance- Google Developers BlogThis post announces the Gemini 2.5 Pro Preview (I/O edition), highlighting its enhanced coding performance, particularly for front-end and UI development, and improvements in code transformation, editing, and agentic workflows. It introduces new capabilities like video-to-code generation and easier feature development by generating UI components from design files. The model is now #1 on the WebDev Arena leaderboard and shows strong performance in junior-dev evaluations. It also addresses developer feedback on function calling errors and trigger rates. The post provides links to Google AI Studio and Vertex AI for access.
  68. Announcing the general availability of Llama 4 as MaaS on Vertex AI- Google Developers BlogThis post announces the general availability of Llama 4 and Llama 3.3 70B as managed API endpoints (Model-as-a-Service) on Google Cloud's Vertex AI. It highlights the benefits of using Vertex AI for deploying these models, including zero infrastructure management, guaranteed performance, and enterprise-grade security. The post provides a Python code example using the OpenAI library to interact with the Llama 4 MaaS endpoint for multimodal requests (text and image). It also touches upon cost and quota considerations.
  69. How It’s Made: Little Language Lessons uses Gemini’s multilingual capabilities to personalize language learning- Google Developers BlogThis post details the development of 'Little Language Lessons', a language learning application powered by Gemini models. It showcases specific prompt engineering techniques for structured JSON output (vocabulary, phrases, grammar tips, dialogue context, speaker messages, descriptors) and the use of Gemini's vision capabilities for object detection with bounding box coordinates. It also mentions the integration of Cloud Translation API and Cloud Text-to-Speech API for enhanced functionality.
  70. Start building with Gemini 2.5 Flash- Google Developers BlogIntroduces Gemini 2.5 Flash, a new model available in preview via Gemini API, Google AI Studio, and Vertex AI. Highlights its hybrid reasoning capabilities, cost-efficiency, and fine-grained controls for managing 'thinking' budgets (0-24576 tokens) to balance quality, cost, and latency. Provides examples of prompts requiring low, medium, and high reasoning, and includes a Python code snippet demonstrating API usage with a thinking budget.
  71. Introducing TxGemma: Open models to improve therapeutics development- Google Developers BlogIntroduces TxGemma, a collection of open models (2B, 9B, 27B sizes with 'predict' and 'chat' versions) fine-tuned from Gemma 2 for therapeutic development tasks like molecule property prediction, classification, regression, and generation. TxGemma-Predict models outperform previous generalist models and specialized models on various therapeutic tasks. TxGemma-Chat models offer conversational capabilities for explaining reasoning and answering complex questions. Demonstrates Agentic-Tx, a therapeutics-focused agentic system powered by Gemini 2.0 Pro, integrating TxGemma and other tools for complex reasoning-intensive chemistry and biology tasks.
  72. Gemini 2.0 Deep Dive: Code Execution- Google Developers BlogThis post announces the general availability of code execution for Gemini 2.0 models in Google AI Studio and the Gemini API. It details how code execution works, including the Python sandbox environment, available libraries (Numpy, Pandas, Matplotlib), execution limits, and file I/O capabilities. It also highlights new experimental features like integration with the Multimodal Live API, Grounding with Google Search, and Gemini 2.0 Thinking Mode. Practical examples demonstrate real-time data analysis and visualization, and solving optimization problems. The post provides links to GitHub for hands-on experience and the Gemini API docs for further exploration.
  73. Data Science Agent in Colab: The future of data analysis with Gemini- Google Developers BlogThis post announces the general availability of the Data Science Agent in Colab for users age 18+ in select countries and languages. It details the agent's workflow (upload data, describe goals in Gemini side panel, agent generates notebook) and benefits (fully functional notebooks, modifiable solutions, sharable results, time savings). It also highlights the agent's 4th place ranking on the DABStep benchmark.
  74. Introducing PaliGemma 2 mix: A vision-language model for multiple tasks- Google Developers BlogIntroduces PaliGemma 2 mix, a multimodal vision-language model that can perform multiple tasks (detection, segmentation, OCR, question answering, captioning) out-of-the-box. It offers different model sizes (3B, 10B, 28B parameters) and resolutions (224px, 448px), and is compatible with Hugging Face Transformers, Keras, PyTorch, JAX, and Gemma.cpp. The post provides examples of its performance on various tasks.
  75. See the Similarity: Personalizing Visual Search with Multimodal Embeddings- Google Developers BlogThis post details the practical application of multimodal embeddings for visual search and enterprise-scale document search. It showcases two implementations: one using Cloud Firestore Vector Search for a smaller dataset (artist's illustrations) and another using Vertex AI Vector Search for a larger dataset (thousands of Google Slides presentations). Key technical details include the use of 1408-dimensional embeddings, storing embeddings in Firestore and Cloud Storage, indexing with Firestore CLI and Vertex AI, and querying using KNN (Firestore) and ScaNN (Vertex AI). It also touches on dimensionality reduction with UMAP for visualization and discusses the trade-offs between KNN and ScaNN for different scales.
  76. Beyond English: How Gemma open models are bridging the language gap- Google Developers BlogThis post details how Google's Gemma open models are being used to build localized LLMs for diverse linguistic communities. It highlights the work of AI Singapore with SEA-LION for Southeast Asian languages and INSAIT with BgGPT for Bulgarian. Key technical contributions include: 1. Leveraging Gemma's multilingual capabilities for improved performance and reduced costs. 2. Addressing the challenge of finding high-quality, relevant training data for underrepresented languages through efforts like Project SEALD. 3. Implementing techniques to filter out irrelevant content and ensure cultural relevance. 4. Using a 9 billion parameter model (Gemma 2-9B) for greater accessibility in throughput-constrained environments. 5. INSAIT's use of continuous pre-training on ~85 billion Bulgarian tokens and a novel 'branch-and-merge' method to mitigate catastrophic forgetting, allowing models to retain English and math proficiency while learning new languages. 6. The Kaggle competition 'Unlock Global Communication with Gemma' encourages further development of localized LLMs.
  77. Introducing PaliGemma 2: Powerful Vision-Language Models, Simple Fine-Tuning- Google Developers BlogIntroduces PaliGemma 2, a new vision-language model that builds on Gemma 2. Key features include scalable performance with multiple model sizes and resolutions, long captioning capabilities, and demonstrated leading performance on specialized tasks like chemical formula recognition and X-ray report generation. It is designed as a drop-in replacement for PaliGemma and offers straightforward fine-tuning.
  78. Web AI Summit 2024 Recap: Client-Side AI for Developers- Google Developers BlogThis post is a recap of the first Web AI Summit, highlighting talks on client-side AI for web developers. Key themes include the use of machine learning models directly in the web browser for low latency, lower costs, and privacy. Specific technologies discussed include Transformers.js, WebNN API, WebLLM for in-browser LLM inference, ml5.js, and Chrome's built-in AI features like the Prompt API. Case studies from industry (CPG, blog services) and the potential of AI in Chrome Extensions were also presented. The summit emphasized the growing capabilities and accessibility of AI for web development.
  79. Supercharging AI Coding Assistants with Gemini Models' Long Context- Google Developers BlogThis post details a collaboration with Sourcegraph to evaluate the impact of Gemini 1.5 Pro and Flash models with 1M token context windows on their Cody AI coding assistant. Key findings include significant improvements in Essential Recall, Essential Concision, and Helpfulness benchmarks, along with a reduction in hallucination rate from 18.97% to 10.48%. It also discusses tradeoffs in first-token latency and mitigation strategies like prefetching and layered context model architecture, achieving ~5 second latency for 1MB contexts.
  80. Gemini API and Google AI Studio now offer Grounding with Google Search- Google Developers BlogIntroduces 'Grounding with Google Search' for Gemini API and Google AI Studio, enabling models to leverage real-time Google Search results for more accurate and fresh responses. This feature provides grounding sources (inline links) and Search Suggestions. It is supported by Gemini 1.5 models and can be enabled via the 'google_search_retrieval' tool. The post also details dynamic retrieval, allowing developers to set a prediction score threshold to control when grounding is applied, balancing accuracy with cost and latency. Includes a Python code example for enabling the tool.
  81. Introducing Keras Hub: Your one-stop shop for pretrained models- Google Developers BlogIntroduces KerasHub, a unified library for pretrained NLP and CV models, consolidating KerasNLP and KerasCV functionalities. Highlights unified developer experience, model publishing, LoRA fine-tuning, quantization, and multi-host training. Provides installation instructions and code examples for using Gemma, PaliGemma, and Stable Diffusion 3. Details the transition for KerasNLP and KerasCV developers, including changes in import statements and model loading APIs. Introduces built-in preprocessing within KerasHub models and the relocation of loss functions to `keras.losses`.
  82. Compare Mode in Google AI Studio: Your Companion for Choosing the Right Gemini Model- Google Developers BlogIntroduces 'Compare Mode' in Google AI Studio, a feature that allows users to evaluate responses from different Gemini and Gemma models side-by-side. This feature helps developers assess response quality, latency, and the impact of system instructions to make informed decisions about model selection for their use cases.
  83. Gemini 1.5 Flash-8B is now production ready- Google Developers BlogIntroduces Gemini 1.5 Flash-8B, a production-ready, smaller, and faster variant of Gemini 1.5 Flash. Highlights its 50% lower price, 2x higher rate limits (4000 RPM), and lower latency on small prompts compared to the previous 1.5 Flash model. Mentions its performance across benchmarks, particularly for chat, transcription, and long context language translation. Details pricing structure and availability via Google AI Studio and Gemini API.
  84. Gemma explained: PaliGemma architecture- Google Developers BlogThis post introduces PaliGemma, a new vision-language model (VLM) architecture. It details the architecture by explaining the SiglipVisionModel (including SiglipVisionEmbeddings, patch_embedding, position_embedding, and encoder) and the PaliGemmaMultiModalProjector, which combines vision and language representations. It also describes the language_model component based on Gemma 2B. The post explains how PaliGemma extends the Gemma tokenizer with tokens for image coordinates and segmentation codewords, enabling object detection and image segmentation. It provides an example of object segmentation output, explaining the coordinate system and the encoding of segmentation masks.
  85. Gemma explained: RecurrentGemma architecture- Google Developers BlogThis post details the RecurrentGemma architecture, a hybrid model that mixes gated linear recurrences with local sliding window attention, based on the Griffin architecture. It explains the trade-offs between transformer and recurrent models for long context prompts, the architectural components (embedding layer, decoder layers with recurrent and MQA blocks), and key parameters like model width, RNN width, MLP expansion factor, and local attention window size. It also provides a Python code snippet for the RecurrentGemmaForCausalLM model.
  86. Gemma explained: What’s new in Gemma 2- Google Developers BlogThis post introduces Gemma 2, a new suite of open AI models available in 2B, 9B, and 27B parameter sizes. It details architectural innovations including alternating local and global attention, logit soft-capping, RMSNorm for normalization, and Grouped-Query Attention (GQA). The post highlights performance advantages of knowledge distillation and GQA over MHA, and the benefits of deeper models. It also provides code examples for Gemma 27B and notes on GQA head configurations for different model sizes.
  87. Gemma explained: An overview of Gemma model family architectures- Google Developers BlogThis post details the architecture of the Gemma model family, including Transformer decoder-only models, variations like CodeGemma, Gemma 2, RecurrentGemma, and PaliGemma, and their core parameters such as d_model, layers, feedforward hidden dims, num heads, num KV heads, head size, and vocab size. It also provides code examples for exploring model architectures.
  88. 3 fun Gemma project ideas- Google Developers BlogThis post details three project ideas using Gemma, an open model family. The first project demonstrates fine-tuning Gemma for translating old Korean literature into contemporary Korean, highlighting the use of SentencePiece for tokenization and LoRA for fine-tuning. It also suggests expansion ideas for American English <-> British English translation and Kansai-ben dialect translation. The second project explores using Gemma for game design brainstorming, showing how it can flesh out game concepts, design characters, and write stories based on a one-sentence pitch. Expansion ideas include generating marketing phrases, florist ideas, and food recipes. The third project outlines using Gemma on a Raspberry Pi for composing magical letters from Santa, though the details are cut short.
  89. Introducing PaliGemma, Gemma 2, and an Upgraded Responsible AI Toolkit- Google Developers BlogIntroduces PaliGemma, an open vision-language model (VLM) built on SigLIP and Gemma, for tasks like image/video captioning, VQA, object detection/segmentation. Announces Gemma 2, a next-generation model with a new architecture for improved performance and efficiency, comparable to Llama 3 70B at half the size and reduced deployment costs. Releases the LLM Comparator as an open-source tool for evaluating model safety and quality side-by-side, expanding the Responsible Generative AI Toolkit.
  90. Introducing new AI tools on Google for Developers- Google Developers BlogIntroduces new AI-powered tools for developers: Chat for interacting with documentation, Code Explain for understanding code samples, and AI Powered Search for faster information retrieval on Google for Developers. These tools are in open beta and require a Google Developer Profile.
  91. Build with Gemini models in Project IDX- Google Developers BlogProject IDX now integrates Gemini models to provide AI features that speed up and refine the end-to-end developer workflow. This includes inline AI assistance for real-time error correction, code suggestions, and auto-completion within any file by pressing Cmd/Ctrl + I. It also introduces a Gemini API template for embedding AI-powered features into applications using the Gemini Pro model, supporting multimodal inputs like images, text, and code, and can be used with the LangChain framework.
  92. How it's Made - Exploring AI x Learning through ShiffBot, an AI experiment powered by the Gemini API- Google Developers BlogThis post details the technical implementation of ShiffBot, an AI experiment for creative coding education. Key technical contributions include: 1. **Gemini API Integration:** Utilizing the Gemini API for natural language processing and response generation. 2. **Prompt Engineering:** Crafting dynamic prompts that incorporate static persona instructions (based on Daniel Shiffman's style), few-shot learning examples, p5.js principles, conversation history, and the user's current code context from the p5.js editor. 3. **Semantic Retrieval:** Implementing semantic retrieval using text embeddings to create a corpus of Daniel Shiffman's educational content (YouTube videos, chapters, transcripts). This allows ShiffBot to surface relevant resources based on user queries. 4. **Chrome Extension Development:** Building an experimental Chrome extension for the p5.js editor to embed ShiffBot within the coding environment. 5. **AI Studio Usage:** Leveraging Google AI Studio for rapid prompt testing and iteration.