Token-Level Length Modeling

This post introduces the Length Value Model (LenVM), a token-level framework for modeling remaining generation length at each decoding step. LenVM formulates length modeling as a value estimation problem, assigning a constant negative reward to each generated token to predict a bounded, discounted return as a proxy for the remaining generation horizon. This provides annotation-free, dense, unbiased, and scalable supervision. Experiments show LenVM significantly improves exact length matching on LIFEBench and enables continuous control over the performance-efficiency trade-off on GSM8K. It also offers interpretable insights into generation dynamics, revealing how specific tokens influence reasoning towards shorter or longer regimes. LenVM supports length control, prediction, and interpretation, suggesting its potential as a general framework for length modeling and a length-specific value signal for future RL training.