Site Reliability and Outage Management
Predictive test selection: A more efficient way to ensure reliability of code changes

Predictive test selection: A more efficient way to ensure reliability of code changes

11/21/2018 · Mateusz Machalica, Alex Samylkin, Meredith Porth, Satish Chandra

What this post added

Introduced a machine learning-based predictive test selection system that uses a gradient-boosted decision-tree model to probabilistically select regression tests for code changes. This system learns from historical code changes and test outcomes to estimate the likelihood of a test finding a regression, allowing it to run a smaller subset of tests (one-third of transitively dependent tests) while catching over 99.9 percent of regressions. The system also addresses test flakiness by aggressively retrying failed tests during training to distinguish true regressions from non-deterministic failures. This contributes to improved reliability and efficiency of the code development process, complementing other tools like Sapienz (automated testing) and Getafix (automated bug-fixing).

Read the original post ↗