Predictive test selection: A more efficient way to ensure reliability of code changes
Nov 21, 2018Mateusz Machalica, Alex Samylkin, Meredith Porth, Satish Chandra

Summary
Introduced a predictive test selection system that uses machine learning (gradient-boosted decision trees) to probabilistically select regression tests for code changes. This system learns from historical code changes and test outcomes to estimate the likelihood of a test detecting a regression, allowing it to run a smaller subset of tests (one-third of transitively dependent tests) while catching over 99.9% of regressions. It also addresses test flakiness by aggressively retrying failed tests during training data collection.
More from the Site Reliability and Outage Management trail
- Lights Out, Systems On: Validating Instant Power Loss Readiness
- DrP: Meta’s Root Cause Analysis Platform at Scale
- Zoomer: Powering AI Performance at Meta’s Scale Through Intelligent Debugging and Optimization
- Diff Risk Score: AI-driven risk-aware software development
- How Meta keeps its AI hardware reliable