Kimi K3 Model Serving

8/1/2026

This post details the technical underpinnings of Kimi K3, including Kimi Delta Attention (KDA) for 1M context, Attention Residuals, and the Stable LatentMoE framework with Quantile Balancing, Per-Head Muon, Sigmoid Tanh Unit (SiTU), and Gated MLA. It also provides practical guidance on using Kimi K3 on Together AI, covering reasoning effort, streaming, vision input limits, structured output with JSON schema, and dynamic tool loading with examples using the Together Python SDK.

7/29/2026

This post announces a strategic partnership with Moonshot AI, making Together AI a launch platform for Moonshot's models, starting with Kimi K3. It details Kimi K3's architecture, including Kimi Delta Attention (KDA) and Attention Residuals, and highlights its performance for long-horizon coding and agentic workflows. The post also describes how Kimi models are served on Together AI's production inference platform (Serverless, Provisioned Throughput, Dedicated Inference) and the capabilities for post-training these models, emphasizing seamless integration between training and serving.

7/26/2026

This post provides a detailed comparative analysis of Kimi K3 and GPT-5.6 Sol on the DeepSWE benchmark. It quantifies performance differences in terms of pass@1, pass@k, coverage, and reliability, and analyzes cost per rollout and per solved task. Crucially, it explores the divergence in failure modes and task domain strengths between the two models, proposing and evaluating a routing strategy (Kimi K3 cascade to GPT-5.6 Sol on failure) that leverages this divergence to achieve a higher overall accuracy (85.6%) and task coverage (95.6%) than either model alone, while also optimizing cost. It also details performance by programming language and task type.

7/24/2026

This post details a comparative analysis of Kimi K3 and Claude Fable 5 on the DeepSWE benchmark, focusing on coding capabilities, cost-effectiveness, and reliability. It highlights Kimi K3's strengths in pass@2 and pass@4 metrics, its significantly lower cost per solved task (2.8x cheaper), and its open-weight nature providing deployment control. The analysis includes detailed breakdowns by programming language and a discussion of model similarity and failure anatomies. The post positions Kimi K3 as a rational default for agentic work due to its balance of reach, performance, and cost.

6/17/2026

This post details an experiment comparing Kimi K2.7 Code with Claude Fable 5 for landing page generation. It highlights Kimi's significantly lower cost (94% less) and competitive quality, especially when provided with multimodal design inspiration via an MCP server. The post also discusses the experimental setup, prompts used, cost breakdowns, and a scoring rubric for comparing output quality, concluding that open-source models are a practical choice for such workflows.

4/28/2026

This post announces the availability of NVIDIA Nemotron 3 Nano Omni on the Together AI platform. It highlights the model's multimodal reasoning capabilities (video, images, audio, language) and its hybrid Mamba-Transformer MoE architecture. The post emphasizes Together AI's research optimizations for high throughput and cost-efficient inference, managed infrastructure for agentic workloads, and secure APIs. It details how Nemotron 3 Nano Omni eliminates fragmented multimodal pipelines, enabling unified context for agent applications and supporting deployment on NVIDIA Hopper and Blackwell architectures.

12/15/2025

Introduces native availability of NVIDIA Nemotron 3 Nano, a hybrid Mamba-Transformer + sparse MoE reasoning model with ~3B active parameters and 1M-token context. Highlights its optimization on Together AI for high throughput and cost-efficiency, making it suitable for agentic systems, coding assistants, scientific agents, tool-using planners, and enterprise context applications. Details performance, reliability, and cost-efficiency benefits of running Nemotron 3 Nano on Together AI, including an OpenAI-compatible interface for easy adoption.

8/27/2025

Introduces DeepSeek-V3.1, a hybrid LLM with configurable modes for fast responses (non-thinking) and deep reasoning (thinking). Details performance benchmarks for both modes, highlighting improvements in complex tasks with thinking mode. Describes built-in agent support (code and search) with optimized tool-calling workflows. Mentions extended context training (32K and 128K tokens). Outlines production deployment on Together AI infrastructure, including technical specs (671B total parameters, 37B active per token, 128K context, MIT license) and infrastructure guarantees (99.9% uptime SLA, SOC 2 compliant). Provides Python SDK example for integration and links to playground and documentation.

8/11/2025

This post introduces and evaluates the gpt-oss-120B model on the Together AI platform. It details a comparative analysis against o4-mini across five practical tests: terminal snake game development, creative SVG generation, advanced instruction following, mathematical reasoning, and web-enhanced information synthesis. The results show gpt-oss-120B outperforming o4-mini in code generation and instruction following, tying in mathematical reasoning and web synthesis, and showing mixed results in creative SVG generation. The post also emphasizes the benefits of open-source models, including customization, cost-efficiency, and deployment flexibility, and highlights Together AI's optimized inference speeds and competitive pricing for these models.

8/5/2025

This post announces the availability of OpenAI's gpt-oss-120B and gpt-oss-20B models on Together AI's infrastructure. It details their availability as serverless and dedicated endpoints, emphasizing performance, economics, and reliability (99.9% uptime SLA). The post highlights developer tooling, including fine-tuning and OpenAI-compatible APIs. It mentions optimizations with NVIDIA, FlashAttention, and custom kernels. Pricing for input/output tokens is provided, along with mentions of the Batch API for cost savings. Real-world applications and a Python inference example are included.

7/25/2025

Introduces Qwen3-Coder-480B-A35B-Instruct, a 480B parameter agentic coding model with 256K native context (1M extrapolated), demonstrating frontier-level performance on SWE-bench Verified, Agentic Coding, Agentic Browser Use, and Agentic Tool Use benchmarks. Highlights its capability to handle entire codebases and complex engineering workflows like legacy system modernization, cross-system feature development, and complex debugging. Emphasizes deployment on Together AI's optimized infrastructure for massive scale, advanced training, and production readiness with 4x faster inference. Provides Python SDK example for integration and mentions availability via Interactive Playground, API Documentation, Batch API, and Fine-tuning access.

7/14/2025

This post announces the availability of the Kimi K2 model on Together AI's platform. It details the model's 1 trillion parameters, Mixture-of-Experts design (8 experts from 384), and training on 15.5 trillion tokens using the MuonClip optimizer. The post highlights Kimi K2's performance benchmarks (EQ-Bench3, Creative Writing, SWE-bench Verified, AceBench, LiveCodeBench v6) and its agentic capabilities with native tool use and autonomous workflows. It also emphasizes the deployment benefits on Together's platform, including serverless access, competitive pricing (e.g., $1.00/1M input tokens), 99.9% availability SLA, multi-region deployment, SOC 2 compliance, and seamless scaling. The post also mentions custom fine-tuning options and provides example API usage with the Python SDK.