How Generative Recommenders Are Redefining RecSys at Scale | NVIDIA Technical Blog

Introduces Generative Recommenders (GRs) as a new architectural paradigm for RecSys, shifting from embedding-similarity to sequence modeling. Details production challenges for GRs at scale, including data volume, sparsity, cold start, and latency. Presents the `recsys-examples` repository with optimized implementations of HSTU and Semantic ID models, featuring DynamicEmb for dynamic embedding tables, a KV cache, fused CUDA kernels, and parallelism support. Introduces `nv-embedding-cache` (NVE) for hierarchical, multi-tier caching of massive embedding tables to enable low-latency inference and high throughput. The post also implicitly links these advancements to the broader trend of token-metered AI service billing.