Inside the feature store powering real-time AI in Dropbox Dash

This post details the evolution of Dropbox's feature store for Dash AI, focusing on architectural choices and performance optimizations. Key contributions include: replacing a Python feature serving layer with a Go service to achieve sub-100ms latency targets, leveraging goroutines and optimized JSON parsing for true concurrency; implementing a three-part ingestion system (batch, streaming, direct writes) to balance feature freshness with computational efficiency, including intelligent change detection in batch ingestion to reduce update times from over an hour to under five minutes; and integrating Feast for orchestration, Spark for computation, and Dynovault (DynamoDB-compatible) for low-latency online serving. The post highlights tradeoffs made in choosing Feast and designing the hybrid architecture to bridge on-premises and cloud environments.