TechPrufer
Sign inGet started

Blogs›Together AI›Provisioned Throughput for LLM Inference

Provisioned Throughput for LLM Inference

Provisioned Throughput for LLM Inference

2
posts
2026

Together AI now offers Provisioned Throughput, a new inference form factor for frontier open models. This provides guaranteed capacity with a 99% uptime SLA and token-based pricing, bridging the gap between best-effort serverless and complex dedicated inference. It offers predictable costs, up to 90% lower than proprietary alternatives, and is available for models like MiniMax M3 and GLM-5.2. Provisioned Throughput Units (PTUs) are consumed based on input, cached input, and output tokens, allowing for efficient serving of models with advanced features like 1M context windows and multimodality. This includes deep performance optimizations such as KV-Block-Major sparse attention kernels, paged attention integration, optimized index scoring kernels, and Rust-based multimodal preprocessing gateways, resulting in significant throughput improvements.

2026

Open, convenient and predictable: Introducing Provisioned Throughput

7/8/2026

Introduced Provisioned Throughput, a new inference capacity model for open LLMs. This feature provides guaranteed token capacity with a 99% uptime SLA, offering predictable pricing and simplifying inference management for production workloads. It defines Provisioned Throughput Units (PTUs) and their consumption rates for input, cached input, and output tokens, enabling users to estimate costs based on traffic shape. The initial offering supports MiniMax M3 and GLM-5.2 models.

Serving MiniMax-M3 for efficient inference: Unlocking 1M-Token Context and Multimodality Without Regrets

6/2/2026

This post details the engineering breakthroughs made by Together AI to efficiently serve the MiniMax M3 model, focusing on optimizations for its 1M-token context window and native multimodality. Key contributions include the development of a KV-Block-Major sparse attention kernel, integration of paged attention for MSA, optimization of the decode index scoring kernel, and the implementation of a Rust-based multimodal preprocessing gateway (SMG). These efforts resulted in 81-125% throughput improvements and addressed challenges related to sparse attention computation, KV cache management, and multimodal processing.