Autonomous testing of services at scale

What this post added

This post details the design and implementation of an integration testing infrastructure for backend services at Meta, along with a new autonomous testing extension. The integration testing infrastructure provides a code framework, test scheduling, and execution capabilities to encourage engineers to write effective tests. It addresses challenges in defining test environments by reusing production infrastructure (containerization, routing) to create isolated, ephemeral entities. The autonomous testing extension borrows from fuzz testing and leverages the homogeneity of the software stack to automatically generate tests. Key aspects include defining test environments with isolation layers, specifying test inputs via test harnesses and mocks, and using custom assertions or generic failure detection as test oracles. The isolation strategy for autonomous testing involves a nuanced approach combining application-level and network-level isolation to allow known read-only traffic, use allowlists for safe destinations, reroute standard RPC traffic to universal mocks, and block all other requests.