TechPrufer
Sign inGet started

Blogs›Datadog›Heatmap Visualization for Distributions

Heatmap Visualization for Distributions

Heatmap Visualization for Distributions

2
posts
2019–2024

Datadog developed a heatmap visualization system to effectively represent high-resolution distribution metrics over time at arbitrary scale. This system leverages DDSketch for aggregating data and a frontend strategy of sending bins of counts to achieve constant-time rendering. The visualization allows users to identify distinct 'modes' or patterns within data distributions that might be obscured by traditional percentile graphs, enabling deeper analysis of system behavior and performance. This post details the design and implementation of DDSketch, a novel quantile sketch algorithm that provides relative-error guarantees and is fully mergeable, addressing the limitations of existing algorithms for large-scale, distributed monitoring data.

2024

How we built the Datadog heatmap to visualize distributions over time at arbitrary scale | Datadog

5/1/2024

This post details the engineering decisions and technical implementation behind Datadog's heatmap visualization for distribution metrics. It explains the use of DDSketch for scalable data aggregation and a frontend approach that limits the number of bins to achieve constant-time rendering. The core contribution is the development of a visualization that reveals distinct data distribution 'modes' over time, offering a more nuanced understanding of system performance compared to traditional percentile graphs.

2019

Computing accurate percentiles with DDSketch | Datadog

9/23/2019

This post introduces DDSketch, a new quantile sketch algorithm designed to accurately compute percentiles from large-scale, distributed monitoring data. It addresses the challenges of exact percentile computation and distributed aggregation by providing a fully mergeable sketch with relative-error guarantees. The post explains the limitations of existing algorithms like GK and details how DDSketch's design enables accurate percentile calculation and efficient data aggregation.