TechPrufer
Get started
Apple
Apple logoApple · Feature trail

Categorical Flow Maps for Language Modeling

1 post · Aug 2026

What this trail covers

Categorical Flow Maps (CFMs) are scaled to 1.7B parameters and trained on 2.1T tokens, demonstrating competitive text generation quality in few inference steps. A likelihood bound for CFMs in the semi-discrete setting is introduced, enabling scoring on standard LM benchmarks. Challenges and insights for training at scale, including loss weighting and time scheduling, are provided.

All Apple trails
  1. Scaling Categorical Flow MapsThis post scales Categorical Flow Maps (CFMs) to 1.7B parameters on 2.1T tokens, achieving diverse, high-quality text generation in as few as 4 inference steps with near-data-level token entropy. It introduces a likelihood bound for CFMs in the semi-discrete setting for scoring on LM benchmarks and provides prescriptive insights on loss weighting and time scheduling for large-scale training.