Blazing fast on-device GenAI with LiteRT-LM- Google Developers Blog

What this post added

This post introduces LiteRT-LM, an optimized runtime for deploying Gemma 4 and other LLMs on edge devices. It details the underlying stack, including LiteRT, XNNPACK, and MLDriift kernels, and highlights performance gains on various hardware backends (CPU, GPU, NPU) and platforms (Android, iOS, Web). Key features include native support for Multi-Token Prediction (MTP) for up to 2.2x speedup, advanced session management for continuity and efficiency, and optimized memory utilization. The post also covers agentic workflow orchestration with Thinking Mode and constrained decoding, and announces new cross-platform interfaces for Swift (iOS) and JavaScript (WebGPU).