Generative Recommender Systems

This feature thread tracks the evolution of Generative Recommender Systems (GRs), a paradigm shift from traditional embedding-based approaches. Initial efforts focused on architectural shifts towards sequence modeling objectives and transformer-like architectures, exemplified by models like Hierarchical Sequential Transduction Units (HSTU) and Semantic IDs, which address challenges in scalability, cold start, and the long-tail problem. The NVIDIA recsys-examples repository provides production-ready examples. Subsequent developments have integrated HSTUs with NVIDIA Dynamo-Triton for low-latency inference, leveraging PyTorch Ahead-of-Time (AOTI) compilation for native C++ artifacts, FlexKV-backed KV caching to avoid recomputing long user histories, and NV embedding cache for efficient memory usage. This enables significant performance improvements, with up to 5.93x lower latency demonstrated for an eight-layer HSTU model at dynamic batch size 8 with a 100% GPU KV-cache hit rate.