Scaling Airbnb’s identity graph with a unified knowledge graph infrastructure

This post details the migration of Airbnb's identity graph from a third-party SaaS graph database to an internally managed knowledge graph infrastructure built on JanusGraph, DynamoDB, and OpenSearch. It highlights key challenges such as scalability (7 billion nodes, 11 billion edges), query complexity (4-8 hops), long-tail latency, and system stability. The post introduces the architecture of the internal graph infrastructure, including optimizations like custom transaction strategies leveraging DynamoDB's conditional writes, parallel query execution for the getMultiSlices interface, and integration with Airbnb's distributed tracing for enhanced observability. It also discusses client-side query optimizations, such as removing Path steps and optimizing side-effect steps, to improve performance. The migration resulted in significant P99 latency reductions, improved system stability by eliminating manual reboots, and a tenfold increase in write QPS scalability.