Scaling Bitbucket's Database

This post details the implementation of a database read/write splitting strategy for Bitbucket. It describes the problem of a stressed primary PostgreSQL database due to high traffic from gu-bb and gu-api services. The solution involves routing read requests to replicas and write requests to the primary, using LSNs to manage replication lag and ensure data consistency. The implementation uses Django middleware, Redis (Elasticache) for LSN storage, and Envoy for connection pooling. Tradeoffs include added latency and operational overhead. Results show a significant reduction in primary database load and maintained added latency.