How Shopify Manages Petabyte Scale MySQL Backup and Restore - Shopify

What this post added

This post details the redesign of Shopify's MySQL backup and restore solution, moving from Percona's Xtrabackup to GCP's Persistent Disk (PD) snapshot feature. Key technical contributions include: building tooling around the GCP API to invoke PD snapshots, deploying this tooling as Kubernetes CronJobs for automated, frequent (every 15 minutes) backups across regions, implementing a framework for snapshot retention policy management, developing a restore process that exports snapshots to new PD volumes, and establishing a backup verification process that exports daily snapshots to PD volumes and runs Kubernetes Jobs to check for instance startup, replication, and InnoDB page-level corruption. Additionally, tooling was created to export snapshots to an offsite location for disaster recovery.