Mastering Edge AI on Raspberry Pi with LiteRT and Gemma- Google Developers Blog

This post introduces LiteRT, a new high-performance on-device inference runtime for edge AI on platforms like Raspberry Pi. It details how Gemma models (specifically Gemma 4 E2B) can be efficiently deployed and run on Raspberry Pi 5, achieving significant token generation speeds and low memory usage through CPU acceleration via XNNPACK and LiteRT-LM. The post also highlights GPU inference capabilities using LiteRT's WebGPU (Vulkan) backend via ML Drift, supporting various computer vision and audio models, including MediaPipe and YOLO. A deep dive into the Reachy Mini robot pipeline demonstrates a dual-processing architecture leveraging LiteRT for concurrent CPU and GPU workloads. The LiteRT CLI is presented as a tool for streamlining the ML development cycle (conversion, quantization, benchmarking, inference) and its integration with AI coding agents is discussed. Finally, the post emphasizes LiteRT's lean binary footprint for IoT devices and provides instructions for installation and running a Gemma model.