Meta

MobileLab: Highly accurate testing to prevent mobile performance regressions

Oct 19, 2018Joel Beales, Jeffrey Dunn
MobileLab: Highly accurate testing to prevent mobile performance regressions
SummaryAI, from the source post

Introduced MobileLab, a new system for highly accurate mobile performance testing designed to prevent regressions as small as 1 percent. MobileLab achieves this by establishing a deterministic testing environment through several key technical innovations: locking CPU and GPU frequencies to fixed settings, using an HTTP proxy to simulate consistent network responses, managing application on-disk state across trials, remounting data directories to RAM disks for consistent disk I/O, resetting device clocks, dismissing crash dialogs, and replacing logcat tailing with direct socket communication for metric reporting. This system has improved confidence intervals by 7x and reduced false positives by 75 percent compared to previous methods.

More from the Internet Latency and Network Performance Analysis trail