Scaling Categorical Flow Maps

This post scales Categorical Flow Maps (CFMs) to a 1.7B-parameter model trained on 2.1T tokens, demonstrating competitive sample quality and near-data-level token entropy in as few as 4 inference steps. It introduces a likelihood bound for CFMs in the semi-discrete setting for scoring on LM benchmarks and provides prescriptive insights on loss weighting and time scheduling for training at scale.