Unlocking Peak Performance on Qualcomm NPU with LiteRT- Google Developers Blog

Introduces the LiteRT Qualcomm AI Engine Direct (QNN) Accelerator, a successor to the TFLite QNN delegate, for on-device ML inference on Qualcomm NPUs. This new accelerator simplifies mobile deployment by abstracting vendor-specific SDKs and SoC fragmentation, offering a unified workflow for AOT or on-device compilation. It achieves state-of-the-art performance for LLMs and GenAI models, with benchmarks showing up to 100x speedup over CPU and 10x over GPU on Qualcomm's latest SoCs. The post details performance gains for models like FastVLM-0.5B, including low time-to-first-token and high token throughput, and provides a 3-step guide for developers to integrate LiteRT QNN acceleration into their Android applications, including AOT compilation with AI Pack export and deployment via Google Play for On-device AI.