LiteRT: Maximum performance, simplified- Google Developers Blog
May 20, 2025Mogan Shieh, Terry Heo, Jingjiang Li

Summary
Introduces LiteRT's MLDrift for enhanced GPU acceleration with optimized tensor layouts, workgroup optimization, and improved data handling. Adds NPU support for MediaTek and Qualcomm, abstracting vendor SDKs and providing unified model/runtime distribution via AI Packs and Feature Delivery. Simplifies hardware acceleration setup with a new CompiledModel API. Enhances performance with seamless buffer interoperability (TensorBuffer) for zero-copy data transfers between hardware memory types and asynchronous execution using OS-level mechanisms (e.g., EGL sync fences) to minimize latency and power consumption by overlapping computations across CPU, GPU, and NPUs.
More from the Coral Edge AI Platform trail
- ML Drift: Next-Gen GPU AI/ML Inference at the Edge- Google Developers Blog
- Bring multimodal semantic search to the edge with EmbeddingGemma 2- Google Developers Blog
- EmbeddingGemma 2: The Developer Guide- Google Developers Blog
- Mastering Edge AI on Raspberry Pi with LiteRT and Gemma- Google Developers Blog
- LiteRT.js, Google's high performance Web AI Inference- Google Developers Blog