On-device GenAI in Chrome, Chromebook Plus, and Pixel Watch with LiteRT-LM- Google Developers Blog

What this post added

Introduces LiteRT-LM, a C++-based LLM inference framework for edge devices, as an open-source project. Details its integration into Chrome, Chromebook Plus, and Pixel Watch for on-device GenAI. Explains the LiteRT-LM architecture (Engine/Session) and optimizations (context switching, session cloning, CoW KV-cache) for multi-feature LLM deployment. Demonstrates a lightweight pipeline assembly for resource-constrained devices like the Pixel Watch. Provides access to the C++ preview and sample code.