Jane Street

Scaling and benchmarking a critical message bus using a new indexing strategy

Oct 5, 2026
Scaling and benchmarking a critical message bus using a new indexing strategy
SummaryAI, from the source post

Introduced a new indexing strategy for Aria's tip recovery, utilizing topic partitions and a min-heap (optimized via AI-driven heap implementation selection) to efficiently filter messages, resulting in a 30% CPU usage reduction. Developed an adaptive splitting algorithm for subtree stores, combining a greedy clustering algorithm with binary search, to intelligently distribute messages based on volume and improve initial recovery performance.