TechPrufer
Get started
Atlassian
Atlassian logoAtlassian · Feature trail

Bitbucket Database Read/Write Splitting

1 post · Oct 2020

What this trail covers

Bitbucket is evolving its database architecture to improve scalability and performance by implementing read/write splitting. This involves routing read operations to replicas and write operations to the primary database, with a mechanism to ensure data consistency by using Write-Ahead Log (WAL) sequence numbers (LSNs) to select up-to-date replicas. This approach aims to significantly reduce the load on the primary database and improve overall system responsiveness.

All Atlassian trails
  1. Scaling Bitbucket's DatabaseThis post details the implementation of a database read/write splitting strategy for Bitbucket. It describes the problem of a stressed primary PostgreSQL database due to high traffic from gu-bb and gu-api services. The solution involves routing read requests to replicas and write requests to the primary, using LSNs to manage replication lag and ensure data consistency. The implementation uses Django middleware, Redis (Elasticache) for LSN storage, and Envoy for connection pooling. Tradeoffs include added latency and operational overhead. Results show a significant reduction in primary database load and maintained added latency.