Aurora

This post introduces Aurora, an open-source, RL-based framework for continuous, online adaptation of speculative decoding models. It addresses the problem of stale draft models in production by learning directly from live inference traces. Key contributions include a serve-to-train flywheel architecture, asynchronous RL formulation, Tree Attention for processing complex branching structures, and demonstrated adaptation to distribution shifts, achieving a 1.25x speedup over static speculators and outperforming offline training from scratch in mixed traffic scenarios.