Introducing Northguard and Xinfra: scalable log storage at LinkedIn
6/25/2025
Introduces Northguard, a new scalable log storage system designed to address the limitations of Kafka at LinkedIn. Details Northguard's data model (records, segments, ranges, topics), its use of log striping for load balancing, and the advantages of its range-based approach over indexed partitions for stream processing. Also introduces Xinfra as a virtualized Pub/Sub layer over Northguard and explains the transition from Kafka.