TechPrufer
Sign inGet started

Blogs›Together AI›Long Context Task Decomposition

Long Context Task Decomposition

Long Context Task Decomposition

3
posts
2026

Together AI now offers native deployment of elite proprietary models like DeepSeek V4 Pro, featuring a 512K-token context window (model-level 1M) for long-context reasoning. This model utilizes a large-scale MoE architecture (1.6T parameters, 49B activated) with hybrid attention (Compressed Sparse Attention and Heavily Compressed Attention) to manage serving costs, memory pressure, KV cache usage, and latency. It supports controllable reasoning modes (Non-Think, Think High, Think Max) to match reasoning depth to task difficulty. Pricing includes transparent serverless options with a 90% cost reduction for reused context via cached input pricing. The platform supports a deployment path from Serverless Inference to Dedicated Infrastructure for reserved capacity and production control, catering to workloads like code agents, document intelligence, and research synthesis.

2026

Serving DeepSeek-V4: why million-token context is an inference systems problem

5/11/2026

This post details the serving implications of DeepSeek-V4's million-token context window, focusing on its hybrid attention design (CSA, HCA, SWA). It explains how V4 compresses the token axis of the KV cache to reduce memory pressure and improve concurrency. The post highlights the need for inference engines to manage multiple KV-cache layouts, adapt prefix caching policies, and consider regime-dependent performance (long-context decode vs. short-context prefill). It also emphasizes that the same model weights require different serving profiles based on workload characteristics (long-context agents, coding agents, short chat, RL rollouts). The key technical contributions are the analysis of V4's architectural impact on KV cache management and the identification of critical benchmarking areas for long-context models.

DeepSeek-V4 Pro now available on Together AI

4/29/2026

This post announces the availability of DeepSeek V4 Pro on Together AI, highlighting its 512K-token context window (model-level 1M) and 1.6T-parameter MoE architecture. It details the hybrid attention mechanisms (Compressed Sparse Attention and Heavily Compressed Attention) used to optimize serving for long contexts, reducing FLOPs and KV cache usage. The introduction of three controllable reasoning modes (Non-Think, Think High, Think Max) allows for task-specific reasoning depth. The post also emphasizes the practical benefits of cached input pricing for repeated long-context queries, offering a 90% cost reduction, and outlines the deployment path from serverless to dedicated infrastructure for production workloads.

Plan, divide, and conquer: How weak models excel at long context tasks

3/26/2026

This post introduces a novel 'Divide & Conquer' framework for handling long context tasks in LLMs. It addresses the 'fog of length' problem by decomposing tasks into smaller chunks processed by worker models, with a manager model aggregating results. The framework is validated experimentally, showing that smaller models using this approach can outperform single-shot large models on long context tasks. Key engineering benefits include cost savings, faster processing via parallelism, and simplified tuning. The post also identifies task types where this approach is most effective and where single-shot processing remains superior.