Building real-world on-device AI with LiteRT and NPU- Google Developers Blog

What this post added

This post details the integration of LiteRT, a cross-platform production-ready framework for on-device AI, with Neural Processing Units (NPUs). It highlights how LiteRT abstracts NPU SDK complexities, enabling CPU, GPU, and NPU acceleration across mobile, desktop, and IoT platforms. Real-world examples from Google Meet, Epic Games (Live Link Face), and Argmax Inc. (Argmax Pro SDK) demonstrate significant performance gains (e.g., 25x larger models, 2x speedup) and power efficiency achieved through NPU acceleration. The Google AI Edge Gallery App now includes NPU support for Gemma models, and the Google AI Edge Portal offers benchmarking services for mobile devices. LiteRT's cross-platform support is extended to industrial edge platforms like Qualcomm Dragonwing™ IQ8 Series and Arduino VENTUNO Q, and to AI PCs via OpenVINO™ integration with Intel® Core™ Ultra processors.