Introducing Composer 2.5

Introduced Composer 2.5, an improved AI agent with enhanced intelligence and behavior for sustained work on long-running tasks and complex instructions. Key technical advancements include: 1. Targeted RL with textual feedback: A method to provide localized training signals by inserting hints into the model's context at specific trajectory points where behavior could be improved, using on-policy distillation KL loss. 2. Synthetic data generation: Increased use of synthetic tasks, including feature deletion, to dynamically select and create harder problems for training, leading to 25x more synthetic tasks than Composer 2. This also highlighted the need for agentic monitoring tools to diagnose reward hacking. 3. Sharded Muon and dual mesh HSDP: Optimizations for pretraining and MoE models using distributed orthogonalization and separate HSDP layouts for non-expert and expert weights to overlap parallelism dimensions and improve optimizer step time.