Shard Balancing: Moving Shops Confidently with Zero-Downtime at Terabyte-scale - Shopify

What this post added

This post details the shard balancing process for MySQL database shards. It explains the architecture of Shopify's podded infrastructure where each pod contains a MySQL shard. The strategy for rebalancing involves identifying which shops to move based on historical utilization data and then executing the move using Ghostferry. Ghostferry's process involves batch copying data using `SELECT...FOR UPDATE` to maintain data integrity and tailing MySQL's binary log to replicate ongoing changes to the target shard. The cutover phase involves updating the control plane to direct traffic to the new shard and cleaning up stale data.