Half-Quadratic Quantization of large machine learning models

This post introduces Half-Quadratic Quantization (HQQ), a novel data-free quantization technique for LLMs. HQQ focuses on minimizing errors in model weights by incorporating a sparsity-promoting loss (l_p-norm) and using a Half-Quadratic solver to optimize zero-point and scaling parameters. This approach achieves significant speed-ups (over 50x faster than GPTQ for Llama-2-70B) and competitive compression quality without requiring calibration data, enabling faster deployment of large models.