Meta

Autonomous testing of services at scale

Oct 20, 2021Paul Marinescu
Autonomous testing of services at scale
SummaryAI, from the source post

This post details the design and implementation of an autonomous testing extension built on top of Meta's existing integration testing infrastructure for backend services. Key contributions include: 1. Reusing production infrastructure (containerization, routing) to build ephemeral, isolated test environments. 2. Implementing a nuanced isolation strategy that allows known read-only traffic, service owner-defined allowlists, rerouting to universal mocks, and blocking all other network requests. 3. Utilizing application-level (API-based) and network-level (IP:port-based) isolation, with stack trace inspection for code-level decisions. 4. Employing LD_PRELOAD over BPF for the isolation layer implementation. 5. Demonstrating how this system enables autonomous testing for one-third of Meta's services.

More from the Site Reliability and Outage Management trail