Building real-world on-device AI with LiteRT and NPU- Google Developers Blog

This post introduces LiteRT, a cross-platform framework for on-device AI that enables CPU, GPU, and NPU acceleration. It abstracts vendor-specific NPU SDK complexities, allowing developers to target diverse silicon with a unified API. Real-world examples from Google Meet, Epic Games, and Argmax Inc. demonstrate significant performance gains (e.g., 25x larger models in Google Meet, 2x speedup from GPU to NPU in Argmax Pro SDK) and power efficiency improvements. LiteRT extends NPU acceleration beyond mobile to industrial IoT and AI PCs through integrations with platforms like Qualcomm Dragonwing™ IQ8 Series and Intel® Core™ Ultra processors via OpenVINO™. The Google AI Edge Portal provides benchmarking services for performance validation. LiteRT supports both JIT and AOT deployment, with open models available on Hugging Face and experimental access to the Google Tensor ML SDK.