QNNPACK: Open source library for optimized mobile deep learning

This post introduces QNNPACK, an open-source library of high-performance kernels optimized for mobile deep learning inference. QNNPACK focuses on quantized neural networks, providing efficient implementations of convolutional and fully connected operations on 8-bit integer tensors. It achieves significant performance gains (up to 2x) on mobile devices by optimizing for low-precision computations and leveraging the PDOT microkernel with fused requantization, enabling it to fit entire panels of input matrices into L1 cache and eliminate unnecessary memory transformations. QNNPACK supports common neural network primitives like grouped and depthwise convolutions, which were previously underutilized on mobile due to performance limitations.