TechPrufer
Sign inGet started

Blogs›Datadog›Go Compression Libraries (czlib and zstd)

Go Compression Libraries (czlib and zstd)

Go Compression Libraries (czlib and zstd)

1
posts
2016

Datadog has developed and released Go bindings for the czlib (zlib wrapper) and zstd compression libraries. These bindings are designed to improve performance for data pipelines that rely on compression, offering faster compression and decompression rates compared to pure Go implementations. The czlib binding is a fork of vitess's cgzip, adapted for zlib wrapping. The zstd binding mimics the zlib interface and exposes advanced features like stream compression and compression levels. Benchmarks demonstrate significant performance gains, especially for larger messages.

2016

Releasing czlib and zstd Go bindings | Datadog

7/11/2016

This post announces the release of Go bindings for czlib and zstd compression libraries. It details the motivation for creating czlib (performance improvement over pure Go zlib) and its origin as a fork of vitess's cgzip. It also describes zstd as a fast compression library and how the Go binding mimics the zlib interface while exposing zstd's advanced features. Performance benchmarks comparing czlib (non-streaming and streaming) and zstd against the standard library's compress/zlib are provided for both small and large messages.