5 ways Facebook improved compression at scale with Zstandard

What this post added

This post details the integration and benefits of Zstandard (zstd) compression at Facebook, focusing on five key areas: large payload backups and package distribution (introducing multithreading, long range mode, and automatic level determination), data warehousing (increasing block size and reducing data transformations), compressed filesystems (SquashFS and Btrfs for faster deployments and tighter storage), and databases (hybrid compression with RocksDB). These enhancements have led to significant improvements in compression ratios, decompression speeds, and reduced resource consumption across these systems.