Unweight: how we compressed an LLM 22% without sacrificing quality

What this post added

Introduced Unweight, a lossless compression system for LLM model weights that reduces model size by 15-22% and saves ~3 GB VRAM. Unweight exploits redundancy in the exponent byte of BF16 weights using Huffman coding, achieving ~30% compression on the exponent stream. It integrates with Cloudflare's Rust-based inference engine and offers four execution pipelines (full decode, exponent-only decode, palette transcode, direct palette) to balance decompression effort and computation complexity based on workload characteristics. This enables running more models on fewer GPUs, making inference cheaper and faster.