TechPrufer
Sign inGet started

Blogs›Datadog›Go Map Memory Optimization

Go Map Memory Optimization

Go Map Memory Optimization

1
posts
2025

Datadog's Go Agent binaries have undergone significant size reductions, decreasing by up to 77% through systematic dependency auditing, targeted refactors, and re-enabling linker optimizations. This effort involved analyzing build tag usage, isolating dependencies into separate packages, and leveraging tools like `go list` and `goda` for dependency graphing. The project also led to contributions to the Go toolchain and improvements for other large Go projects like Kubernetes. This post details how Go 1.24's Swiss Tables implementation, a new hash table design, significantly reduced memory usage in large in-memory maps by optimizing bucket structure and reducing overhead, leading to fleet-wide savings.

2025

How Go 1.24’s Swiss Tables saved us hundreds of gigabytes | Datadog

7/17/2025

This post details how Go 1.24's new Swiss Tables implementation for hash tables led to significant memory savings in Datadog's Go services. It explains the previous bucket-based map structure in Go 1.23, including bucket layout, slot management, overflow buckets, and map growth strategies. It then contrasts this with the improvements brought by Swiss Tables, which reduce memory overhead per entry and optimize bucket utilization. The post also covers how Datadog profiled and sized these memory savings, identifying specific maps like `shardRoutingCache` and detailing struct-level optimizations that further enhanced efficiency.