Halving the Time: How Uber Eats Rebuilt Its Search Pipeline

This post details how Uber Eats rebuilt its search pipeline, cutting latency in half by optimizing UX (pagination, server-side caching, async rendering), retrieval (removing low-yield lexical strategies, earlier deduplication, product-level embeddings), feature population and ranking (splitting hydration, eliminating false dependencies, request hedging, GPU model serving, relevance-model cache), ads serving (column-oriented data layout, in-memory data, eliminating redundant serialization), and infrastructure tuning (parallel encoding/decoding, reduced embedding precision, multiple network connections, value types for GC reduction). A significant contribution was the use of an agentic AI loop for automated latency problem identification and fixing, validated by an LLM-assisted evaluation framework.