Dropbox

Future-proofing our metadata stack with Panda, a scalable key-value store

Nov 3, 2022Preslav Le
Future-proofing our metadata stack with Panda, a scalable key-value store
SummaryAI, from the source post

Introduced Panda, a petabyte-scale transactional key-value store, as a new layer between Edgestore and sharded MySQL to address scalability and cost-effectiveness issues. Panda abstracts MySQL shards, supports ACID transactions, enables incremental capacity expansion, and allows for rebalancing of data. This architecture separates application and storage concerns, enabling independent iteration and potential unification of backends like Edgestore and Filesystem. Explored and rejected FoundationDB, Vitess, and CockroachDB due to limitations in scale, transaction support, consistency guarantees, and replication models.

More from the Magic Pocket Storage Efficiency trail