TechPrufer
Get started
Together AI
Together AI logoTogether AI · Feature trail

Long Context Task Decomposition

6 posts · Mar 2026 – Aug 2026

What this trail covers

Together AI now offers native deployment of elite proprietary models like DeepSeek V4 Pro and MiniMax M3, featuring up to 1.05M-token context windows for long-context reasoning and advanced software engineering capabilities. This post details the engineering breakthroughs for serving MiniMax M3 efficiently, including a KV-Block-Major sparse attention kernel, paged attention integration for MSA, optimized index scoring, and a Rust-based multimodal preprocessing gateway, resulting in 81–125% throughput improvements. DeepSeek V4 Pro is now available with a 512K-token context window, with full 1M context available on dedicated infrastructure. The model utilizes a large-scale MoE architecture (1.6T parameters, 49B activated) and offers controllable reasoning modes (Non-Think, Think High, Think Max). Cached input pricing offers a significant cost reduction for repeated context usage.

All Together AI trails
  1. DeepSeek V4 Pro 0813 vs GPT-5.6 Sol on DeepSWE: Cost, Coding, and RoutingThis post introduces a comparative analysis of DeepSeek V4 Pro 0813 and GPT-5.6 Sol on the DeepSWE benchmark, focusing on cost, coding performance, and routing strategies. It details performance metrics like pass@1, pass@k, cost per rollout, and failure modes for each model. A key contribution is the demonstration of a cascaded routing strategy where DeepSeek V4 Pro 0813 is used as a first-stage model, escalating to GPT-5.6 Sol only when necessary. This strategy achieves 83.0% task completion at $3.35 per task, outperforming GPT-5.6 Sol alone (72.7% at $8.37) and an oracle router (80.8%). The analysis highlights DeepSeek V4 Pro 0813's strengths in cost-effectiveness and reach with multiple attempts, while GPT-5.6 Sol excels in single-shot precision and speed. The post also details domain-specific and language-specific performance differences and provides a comprehensive data table of results.
  2. DeepSeek V4 Pro 0813 vs Claude Fable 5 on DeepSWE: Cost, Coding, and RoutingThis post introduces a detailed comparison between DeepSeek V4 Pro 0813 and Claude Fable 5 on the DeepSWE benchmark, focusing on cost, coding performance, and routing strategies. It quantifies the performance differences (pass@1, pass@k), cost per rollout, and cost-effectiveness (solves per $100). It also analyzes failure modes, domain-specific strengths, and language-specific performance. A key contribution is the demonstration of a cascaded routing strategy (DeepSeek V4 Pro 0813 first, then Claude Fable 5 if needed) that achieves 82.7% task completion at $8.28 per task, significantly outperforming Claude Fable 5 alone at $21.63 per task. The analysis reveals a low per-task correlation (0.39) between the two models, indicating complementary strengths that make routing effective.
  3. Serving MiniMax-M3 for efficient inference: Unlocking 1M-Token Context and Multimodality Without RegretsThis post details the engineering optimizations for serving the MiniMax M3 model, focusing on its 1M-token context window and native multimodality. Key contributions include the development of a KV-Block-Major sparse attention kernel to reduce attention computation cost, integration of this sparse attention with paged attention for efficient KV cache management, optimization of the decode index scoring kernel, and the implementation of a Rust-based multimodal preprocessing gateway (SMG) for CPU-based vision preprocessing. These optimizations resulted in 81-125% throughput improvements.
  4. Serving DeepSeek-V4: why million-token context is an inference systems problemThis post details the inference systems challenges and solutions for serving DeepSeek-V4, a model with a 1M-token context window. It focuses on the architectural shift to a serving-systems problem due to V4's Compressed Sparse Attention (CSA) / Heavily Compressed Attention (HCA) / Sliding Window Attention (SWA) design. Key contributions include: analysis of KV cache compression on the token axis to reduce memory pressure and improve concurrency; the requirement for multiple KV-cache layouts (CSA, HCA, SWA) and the associated memory management challenges; prefix caching becoming a policy decision across different cache objects; the regime-dependent performance of V4 (long-context decode vs. short-context prefill); the need for different serving profiles for the same weights based on workload shape (long-context agents, coding agents, short chat, RL rollouts); and recommendations for benchmarking V4 based on context-length regime, prefix reuse, cache policy, and endpoint profile.
  5. DeepSeek-V4 Pro now available on Together AIThis post announces the availability of DeepSeek V4 Pro on Together AI, highlighting its 512K-token context window (with 1M model-level context) and 1.6T-parameter MoE architecture. It details the controllable reasoning modes (Non-Think, Think High, Think Max) and their use cases, along with benchmark results. A key technical contribution is the explanation of cached input pricing, which offers a 90% cost reduction for reused context, and its application in workload patterns like code agents and document intelligence. The post also outlines the deployment path from serverless to reserved capacity for long-context workloads, emphasizing the engineering considerations for managing throughput, concurrency, latency, KV cache pressure, and cost.
  7. Plan, divide, and conquer: How weak models excel at long context tasksIntroduces a 'Divide and Conquer' framework for long-context LLM tasks, enabling smaller models to outperform larger single-shot models. The framework addresses 'Model Noise' (superlinear confusion), 'Task Noise' (cross-chunk dependency), and 'Aggregator Noise' (manager failure). Engineering benefits include reduced cost (using cheaper models), faster processing (parallel workers), and easier tuning (predictable noise curves). The framework is best suited for tasks with moderate cross-chunk dependency, like QA and summarization, and less so for tasks requiring subtle long-range connections.