TechPrufer
Get started
LinkedIn
LinkedIn logoLinkedIn · Feature trail

Generative Recommender Training Efficiency

2 posts · May 2026 – Aug 2026

What this trail covers

LinkedIn rebuilt its Follows Recommendation system using LLM-based semantic retrieval and ranking. This new system encodes member and creator profiles into embeddings, enabling topical relevance matching over popularity. It includes offline and online embedding-based retrieval pipelines and integrates these embeddings into downstream ranking models through supervised projection layers. The system has shown significant lifts in follow rates, especially for new members. Further optimizations include engineering techniques to optimize the training efficiency of Generative Recommenders (GRs) at LinkedIn, focusing on data pipeline, compute, and training lifecycle optimizations. Optimizations include a C++-based fused data loader, improved attention kernels (FlashAttention-3, FlexAttention), fused metrics calculation, fused optimizer, and fused embedding table lookups. These optimizations reduced end-to-end GPU hours by up to 65% in internal production training workloads.

All LinkedIn trails
  1. The training infrastructure behind AI-Powered Job Search: 8X faster multi-teacher distillationThis post details the engineering effort behind optimizing the training infrastructure for AI-Powered Job Search, specifically focusing on multi-teacher distillation. It describes the challenges of coordinating multiple specialized teacher models (relevance, engagement, embedding) with a student Small Language Model (SLM) for low-latency serving. Key contributions include: accelerating teacher fine-tuning through PyTorch FSDP2, HSDP, Liger kernels, and H200 GPUs, which doubled throughput and doubled hardware FLOPs utilization; and enabling online distillation by switching to high-throughput inference engines with continuous batching and paged attention for teacher inference, which made teacher inference 2X faster and allowed for 4X larger batches. The post also discusses tensor parallelism for larger teachers and prefetching to overlap computation.
  2. Faster than Light: Optimizing Generative Recommender Training Efficiency at LinkedInThis post details engineering techniques to optimize the training efficiency of Generative Recommenders (GRs) at LinkedIn, focusing on data pipeline, compute, and training lifecycle optimizations. Optimizations include a C++-based fused data loader, improved attention kernels (FlashAttention-3, FlexAttention), fused metrics calculation, fused optimizer, and fused embedding table lookups. These optimizations reduced end-to-end GPU hours by up to 65% in internal production training workloads.