Distill the LLM, Don't Serve It: Search & Personalization at DoorDash — Raghav Saboo, DoorDash

This post details DoorDash's approach to search and personalization by "distilling" LLM reasoning offline into smaller models for serving. It introduces four key primitives: 1) LLM-generated relevance labels to create a golden dataset, improving retrieval NDCG by 2.3%. 2) Semantic IDs, a learned catalog taxonomy that powers query reformulation and achieved 4-5% MRR gains in ranking. 3) Consumer memory stored across three timescales (text, vectors, graph) and represented as context graphs. 4) Steerable, LLM-generated personalized collections, which lifted order rate by nearly 1% in the pets category. The core technical contribution is the pattern of offline LLM reasoning and online distillation for efficient, semantically rich marketplace discovery.