
10/19/2018 · Joel Beales, Jeffrey Dunn
What this post added
Introduced MobileLab, a new system for highly accurate mobile performance testing designed to prevent regressions as small as 1 percent. MobileLab achieves this by establishing a deterministic testing environment through several key technical innovations: locking CPU and GPU frequencies to fixed settings, using an HTTP proxy to simulate consistent network responses, managing application on-disk state across trials, remounting data directories to RAM disks for consistent disk I/O, resetting device clocks, dismissing crash dialogs, and replacing logcat tailing with direct socket communication for metric reporting. This system has improved confidence intervals by 7x and reduced false positives by 75 percent compared to previous methods.