Accelerating on-device AI: A look at Arm and Google AI Edge optimization- Google Developers Blog

This post details the integration of Arm Scalable Matrix Extension 2 (SME2) with Google AI Edge's LiteRT, XNNPACK, and Arm KleidiAI to accelerate on-device AI inference, specifically for generative AI models like audio generation. It outlines a Convert → Optimize → Deploy pipeline using LiteRT-Torch for PyTorch to .tflite conversion, Model Explorer for identifying quantization-safe layers, AI Edge Quantizer for FP32 to INT8 optimization, and LiteRT with XNNPACK/KleidiAI for runtime acceleration. Benchmarks show over 2x reduction in audio generation time and a 4x decrease in memory usage for the DiT submodule, while maintaining perceptual audio quality.