Cart Assistant: Agentic Grocery Shopping on Uber Eats

This post details the system architecture of Uber Eats' Cart Assistant, an AI-powered feature that generates draft grocery carts from natural language or image prompts. It introduces a multi-prompt state graph architecture where each step (Cart Plan Generation, Candidate Retrieval, Semantic Relevance Judging, Price and Deal Constraint Enforcement, Quantity Selection, Guardrails, Cart Assembly, Content Refinement) has a focused responsibility. Key technical contributions include: 1. A structured output schema for the Cart Plan, separating retrieval language from reasoning context and capturing item-level and cart-level constraints. 2. A semantic relevance judging step with a constrained rubric to distinguish direct matches from substitutes. 3. A dedicated Quantity Selection stage combining LLM reasoning with deterministic calculations to map colloquial quantities to purchasable units. 4. A robust guardrail system (deterministic and LLM-based) to enforce boundaries and ensure safe execution. 5. Strategies for parallelism and latency reduction using dynamic asynchronous execution and mixed model sizes. 6. An evaluation-driven development framework for reliable measurement and improvement of the stochastic LLM system.