Reimagining LinkedIn’s search tech stack

What this post added

This post details the engineering effort behind reimagining LinkedIn's search tech stack with LLMs. It introduces a semantic search infrastructure that uses query understanding modules, embedding-based retrieval (EBR) on GPUs, and a Cross-Encoder Small Language Model (SLM) for ranking. Technical innovations include optimizing LLM inference with score caching, ranking-depth control, and traffic shaping. A hybrid inference pipeline using Spark, Flyte, and Flink generates features and job representations. The post also describes a novel approach to product policy relevance measurement using LLM judges, including prompt engineering, distillation into smaller models, and continuous quality monitoring workflows. The EBR model is trained using a dual-tower architecture with a contrastive InfoNCE loss and a margin-based ranking loss, incorporating hard positives and negatives.