TechPrufer
Get started
Cloudflare
Cloudflare logoCloudflare · Feature trail

Optimized Compression Libraries

1 post · Jul 2015

What this trail covers

Cloudflare has optimized compression libraries, including a fork of zlib, to improve performance and compression ratios. This fork incorporates modern C practices, improved hash functions, hardware instructions (iSCSI CRC32, PLCMULQDQ), SIMD instructions, and optimized longest-match functions. Benchmarks show significant speedups and better compression rates compared to standard zlib and other implementations, making it suitable for applications like DNA sequencing file compression and image optimization.

All Cloudflare trails
  1. Fighting Cancer: The Unexpected Benefit Of Open Sourcing Our CodeThis post details Cloudflare's optimized fork of the zlib compression library. Key improvements include using `uint64_t` as the standard type, employing the iSCSI CRC32 hardware instruction for hashing, searching for matches of at least 4 bytes, using SIMD instructions for window rolling, utilizing the `PLCMULQDQ` instruction for CRC32 checksums, and optimizing the longest-match function. An experimental branch with an improved linked list implementation is also discussed. Benchmarks demonstrate superior performance and compression rates compared to other zlib implementations.