Meta

LogDevice: a distributed data store for logs

Aug 31, 2017Mark Marchukov
LogDevice: a distributed data store for logs
SummaryAI, from the source post

Introduced LogDevice, a distributed data store specifically designed for logs. LogDevice provides highly available and durable record storage with repeatable total ordering, addressing challenges in data pipelines, indexing, and distributed transactions at Facebook's scale. Key features include record-oriented append-only storage with trimming, non-deterministic record placement for improved write availability and load balancing, and a sequencer-based architecture that decouples record ordering from storage.

More from the Data Warehousing and Analytics Platform trail