Site Reliability and Outage Management
FOQS: Making a distributed priority queue disaster-ready

FOQS: Making a distributed priority queue disaster-ready

1/18/2022 · Dillon George, Jasmit Kaur Saluja

What this post added

This post details the migration of FOQS from a regional to a geo-distributed, global architecture to enhance disaster readiness. Key technical contributions include: 1. Architectural modifications to support routing and seamless failover of MySQL primaries across regions using global application primitives and Shard Manager. 2. Introduction of an intelligent routing service with an enqueue distributor for optimal data placement based on logical region preferences and a dequeue distributor for efficient item discovery. 3. Successful migration of tenants at scale with zero downtime for internal clients, addressing challenges like stuck queue items and high cross-region latencies inherent in the previous regional architecture.

Read the original post ↗