SilverTorch: Index as Model — A New Retrieval Paradigm for Recommendation Systems

What this post added

Introduced SilverTorch, a unified model-based retrieval system for recommendation engines, replacing a microservice-based architecture with an 'Index as Model' paradigm. This involves reimplementing all retrieval components (ANN search, eligibility filtering, scoring) as pure PyTorch modules within a single neural network. Key technical contributions include the Bloom index filter for efficient GPU-based filtering and fused Int8 ANN search for reduced memory usage and improved serving performance, enabling higher throughput and cost efficiency.