How bitdrift scaled to 121 million concurrent gRPC connections on Amazon CloudFront for live telemetry sporting events | Amazon Web Services

What this post added

This post details how bitdrift scaled to handle 121 million concurrent gRPC connections for live telemetry sporting events by migrating from Route 53 Weighted routing to Multi-Value Answer routing. It explains the 'DNS resolution imbalance' and 'TTL-driven traffic concentration' issues caused by Weighted routing, especially with persistent gRPC connections. The solution involved changing the Route 53 routing policy to Multi-Value Answer, which returns multiple IP addresses per DNS query, allowing CloudFront edge nodes to distribute connections across multiple NLBs immediately. This resulted in zero server-side errors during peak traffic surges, a 99.96% reduction in peak 5xx error rate, and a 100% elimination of server-side outages, all achieved with a minimal customer-side change (DNS configuration update only).