How to train custom EAGLE-3 heads for speculative decoding

This post details the integration and training process for EAGLE-3 speculative decoding heads within Baseten's AI model serving platform. It provides a technical deep dive into the EAGLE-3 method, including concepts like TTT-length and number of draft tokens, and offers specific guidance on dataset preparation (regenerating outputs with the target model), hyperparameter tuning (learning rate scaling with model size, greedy decoding preference), training workflow steps, monitoring metrics (loss, accuracy), evaluation benchmarks (1.5-2.5x latency improvement), deployment considerations, and debugging common training and inference issues. The contribution is the practical application and explanation of how to train and leverage custom EAGLE-3 heads for significant LLM inference latency reduction.