Unweight: how we compressed an LLM 22% without sacrificing quality

Introduced Unweight, a lossless compression system for LLM model weights that achieves 15-22% size reduction by compressing only the exponent bytes of BF16 weights using Huffman coding. Implemented four execution pipelines (full decode, exponent-only decode, palette transcode, direct palette) to optimize decompression and inference based on workload characteristics and batch sizes. Decompression occurs in fast on-chip shared memory to avoid HBM bandwidth bottlenecks. Open-sourced GPU kernels and published a technical paper.