TechPrufer
Get started
Discord
Discord logoDiscord · Feature trail

Experiment Metrics Optimization

1 post · Apr 2026

What this trail covers

Discord is optimizing its experiment metrics by reducing the number of metrics used to improve the quality of insights and reduce false positives. This involves a shift from a broad 'Default Metric List' to a curated set of high-quality metrics that capture distinct concepts, addressing issues related to p-values, false positives, and recall in experiment analysis. The team is moving towards using fewer, higher-quality metrics that capture distinct concepts to mitigate issues with false positives and recall.

All Discord trails
  1. Measure Less to Learn More: Using Fewer, Higher-quality Metrics to Capture What MattersThis post details Discord's journey to address the issue of having too many experiment metrics. It highlights the tradeoff between leaving p-values as-is (leading to potential false positives) and adjusting p-values (leading to worse recall). The core contribution is the conclusion that the best solution is to use fewer, high-quality metrics that capture distinct concepts.