Using small language models to serve more relevant DoorDash search ads - DoorDash

This post details the evolution of DoorDash's search relevance prediction system by introducing the use of small language models (SLMs) for low-latency online prediction. It describes the teacher-student paradigm where a fine-tuned LLM generates offline relevance labels (0-1-2 scale) and an SLM learns from these labels. The production model utilizes a DistilBERT bi-encoder with shared weights, CLS pooling, and 64-dimensional embeddings, with an online bilinear layer for scoring. The training process was optimized using the CORAL framework for ordinal regression, achieving 97% validation accuracy. The system is integrated into the serving pipeline to filter irrelevant ads and improve user experience.