SilverTorch: Index as Model — A New Retrieval Paradigm for Recommendation Systems

This post introduces SilverTorch, a novel retrieval paradigm for recommendation systems that unifies all retrieval components (item index, eligibility filter, scoring layer, user tower) into a single neural network, termed 'Index as Model'. This moves away from a microservice mesh architecture to a pure PyTorch module-based system. Key technical contributions include: redesigning retrieval primitives for GPU execution and model graph integration, implementing a Bloom index filter within the model for efficient, parallelized eligibility checks, and developing a fused Int8 ANN search kernel that reduces memory usage and enables returning a larger pool of candidates. These changes result in significantly higher throughput (23.7x), improved TCO efficiency (20.9x), and better recommendation accuracy by enabling increased modeling complexity within sub-100ms latency budgets.