How we could save petabytes of cache storage with Zstandard and Pingora

Introduced Cache Transcoding, a system that compresses eligible assets (compressible text, >4KiB, no existing Content-Encoding) using Zstandard (level 3) before storing them in cache. This reduces on-disk size by an average of 2.834x, saving storage and bandwidth at the cost of a small CPU increase (4.31 ns/byte encode, 1.56 ns/byte decode). The system preserves original content length and decodes before serving to the client or transferring between cache tiers. Eligibility checks prevent transcoding already compressed media or small assets.