Accelerating on-device AI: A look at Arm and Google AI Edge optimization- Google Developers Blog

What this post added

This post details the integration of Arm Scalable Matrix Extension 2 (SME2) with Google AI Edge (LiteRT, XNNPACK, KleidiAI) to accelerate on-device AI inference on Arm CPUs. It showcases a workflow for converting PyTorch models to .tflite using LiteRT-Torch, optimizing them with Model Explorer and AI Edge Quantizer (demonstrating INT8 dynamic quantization for DiT submodule), and deploying them for high-performance inference via LiteRT's XNNPACK delegate. The post highlights over 2x speed improvements and 4x memory reduction for Stable Audio Open, while maintaining audio quality.