Performance Testing At Scale—for BFCM and Beyond - Shopify

What this post added

This post details Shopify's approach to BFCM Scale Testing, a critical exercise for preparing the e-commerce platform for peak traffic. It outlines a five-step process: scaling up core platform and subsystems, running load tests simulating BFCM traffic, scaling down, analyzing results, and iterating on improvements. The post elaborates on the scaling-up strategy, including coordinating with cloud vendors and scaling upstream/downstream components. It describes five distinct load test flows (browsing/buying, admin, flash sale, Storefront API, Hydrogen/Oxygen) executed by the Cronograma tool. The methodology for determining test load, including using projections and testing regional failover, is explained. The importance of scaling down in reverse order of scale-up to protect dependencies is highlighted. Finally, the post discusses the blameless retrospective process for analyzing results and iterating on the plan, contrasting full-scale testing with scaled-model testing and emphasizing the benefits of the former for uncovering complex scaling issues.