TechPrufer
Get started
Together AI
Together AI logoTogether AI · Feature trail

Coding Agent Training Dataset

2 posts · Jul 2025 – Feb 2026

What this trail covers

Together AI continues to advance agentic coding capabilities with the release of Qwen3-Coder-480B-A35B-Instruct. This model demonstrates frontier-level performance on complex, real-world software engineering workflows, including legacy system modernization, cross-system feature development, and complex debugging. Its 480B parameters and 256K native context window (extensible to 1M) allow it to hold entire codebases in memory and autonomously execute intricate tasks. Together AI's optimized infrastructure provides production-ready deployment with zero setup, instant accessibility, and significantly faster inference, enabling developers to tackle previously intractable problems.

All Together AI trails
  1. CoderForge-Preview: SOTA open dataset for training efficient coding agentsThis post introduces CoderForge-Preview, the largest open dataset of coding agent trajectories (258k test-verified trajectories) to date, enabling fine-tuning of models like Qwen-3 32B to achieve 59.4% pass@1 on SWE-Bench Verified. It details the data generation process from R2E-Gym, SWE-Smith, and SWE-Rebench, including setup with OpenHands v0.52.1 and Qwen3-Coder-480B for generation. The post compares CoderForge-Preview to other datasets, highlighting its scale and long-context coverage. It also details the training setup using FSDP & Sequence Parallelism with Ulysses and multi-packing for 128K sequences, and a token-level loss formulation. The dataset is licensed under permissive open-source licenses.
  2. Qwen3-Coder: The Most Capable Agentic Coding Model Now Available on Together AIThis post announces the availability of Qwen3-Coder-480B-A35B-Instruct on Together AI. It highlights the model's capabilities in agentic coding, SWE-bench Verified, and agentic browser/tool use, showcasing its superiority over other open models and its parity with Claude Sonnet 4. The post emphasizes the model's ability to handle complex, interconnected engineering tasks across entire codebases, such as legacy system modernization, cross-system feature development, and complex debugging. It also details the underlying infrastructure optimizations on Together AI that enable the production-ready deployment of such massive models, including massive scale (480B parameters, MoE efficiency), advanced training (7.5T tokens, 70% code ratio), and production readiness (zero setup, instant deployment, 4x faster inference).
  3. DeepSWE: Training a Fully Open-sourced, State-of-the-Art Coding Agent by Scaling RLIntroduces DeepSWE-Preview, a reasoning-enabled coding agent trained from Qwen3-32B using only reinforcement learning (RL). Achieves 59% on SWE-Bench-Verified with test-time scaling, reaching SOTA for open-weight coding agents (42.2% Pass@1, 71.0% Pass@16). Details the training recipe using Agentica's rLLM framework, including scalable dataset curation with R2E-Gym, environment setup (tools, reward function), and Kubernetes integration for scalable agent rollout collection. Enhances GRPO to GRPO++ for stable training and improved performance, incorporating techniques like Clip High, No KL Loss, No Reward Standard Deviation, Length Normalization, Leave One Out, Compact Filtering, and No Entropy Loss. Introduces Compact Filtering to mask trajectories that hit max context, max steps, or timeout.