Engineering VP Josh Clemm on how we use knowledge graphs, MCP, and DSPy in Dash

This post details Dropbox Dash's context engine, which ingests data from third-party apps, performs content understanding (including multimodal analysis for videos), models information as a knowledge graph, and stores it in secure data stores (lexical index with BM25 and vector store) for hybrid retrieval. It discusses the trade-offs between federated and indexed retrieval, highlighting the benefits of indexed retrieval for company-wide access and pre-processing. The post also addresses challenges with MCP (Model Context Protocol) tool definitions consuming context window space and proposes solutions like a 'super tool' for index-based retrieval, knowledge graph modeling to reduce token usage, storing tool results locally, and using sub-agents for complex queries. It elaborates on the value of knowledge graphs for modeling relationships across apps and providing canonical IDs for entities like people, which aids in relevance and retrieval.