Kimi K3 Model Deployment and API

7/30/2026

This post details experiments comparing LoRA and Full Parameter Fine-Tuning (FullFT) on Qwen3.5-9B, focusing on three key areas that influence performance: data coverage, optimization (learning rate tuning), and adapter capacity (rank). It provides a framework for engineers to test these factors before deciding to switch from LoRA to FullFT, demonstrating that data diversity and learning rate optimization can often close performance gaps previously attributed solely to adapter size. Specific experiments on 'Placement', 'Register Allocation', and 'Nexa VM' tasks illustrate these findings.

7/29/2026

This post introduces a practical and cost-effective method for fine-tuning LLM-based embedding models on the Fireworks platform. It details the use of contrastive learning with In-Batch Negatives and InfoNCE loss to adapt general-purpose embedding models (like Qwen3-Embedding-8B) to specific domains. The post presents experimental results showing significant improvements in retrieval tasks (e.g., +36% nDCG@10 on legal citation retrieval) and discusses three training modes: full-parameter updates, LoRA adapters, and a hybrid approach, emphasizing the benefits of adapting existing models over training from scratch.

7/27/2026

This post details the technical implementation of Kimi Delta Attention, FP4 MoE kernels, and adapted decode kernels to optimize Kimi K3 performance. It highlights the model architecture's new Kimi Delta Attention and refined attention residuals, and the scaling of Mixture of Experts (MoE) to 16 out of 896 experts with Stable Latent MoE, resulting in 2.5 times the scaling efficiency compared to Kimi K2. The post also emphasizes the cost-effectiveness for high-volume production workloads and the benefits for large, long-horizon tasks.

7/26/2026

Launched Fireworks Nexus, a product designed to provide engineering organizations with a managed layer of open-weight AI models. Key technical components include: 1. Enterprise Controls & Cost Observability: Centralized control over AI usage, budget tracking, ROI analysis, and policy enforcement, built on a production inference platform with US-hosted endpoints, zero data retention, and global data center presence. 2. Workflow Continuity: Introduced FireConnect, a one-line install tool that maps models based on harness configurations, ensuring compatibility with existing tools (e.g., Claude Code, Codex, OpenCode) and leveraging Fireworks Serverless APIs (Anthropic and OpenAI compatible). FireConnect is open-sourced under Apache 2.0. 3. Intelligent Traffic Management and Migration: Developed a custom routing endpoint (FireRouter) that uses a custom-trained model to score request difficulty. Routine tasks are routed to cost-effective open-weight models served by Fireworks, while difficult tasks are passed through to existing providers using the user's key. This aims for 3-5x cost reduction. Currently routes between Claude Opus 5 and GLM 5.2, or K3 and GLM 5.2.

7/26/2026

This post details the introduction of Fireworks Serverless Training, a new capability that allows for cost-efficient LoRA fine-tuning and deployment of large models like Kimi K3. It highlights the pay-per-token training model, flexible serving options (live merge and Multi-LoRA), and the ability to train adapters for specific tasks like objective learning (Countdown) and step-by-step problem-solving (Frozen Lake). The post also emphasizes the guaranteed numerics alignment between training and inference.

7/26/2026

This post details Trilogy's playbook for using Kimi K3 in cybersecurity workloads, emphasizing the benefits of open-weight models for defensive AI, the practical reference stack used, and how Fireworks provides a managed inference layer with an OpenAI-compatible API for integrating Kimi K3 into production workflows. It highlights the separation of deterministic tools and model reasoning, the importance of bounded inputs and verification, and the flexibility of deployment options from managed endpoints to dedicated capacity.

7/21/2026

This post provides a detailed technical comparison of Kimi K3 against Fable across various task types (SWE, Terminal, Algorithmic, Multi-Language, Legal). It quantifies the performance differences, analyzes cost implications based on token usage and task complexity, and demonstrates the benefits of a routing strategy for optimizing cost and quality. Specific metrics include task solve rates, turns per task, and token counts, illustrating how Kimi K3's cost-effectiveness is achieved through efficient processing and prompt caching, even when handling more tokens.

7/20/2026

This post details the technical collaboration with Heidi Health, focusing on achieving superior frontier model performance. Key contributions include: 1. Demonstrating Supervised Fine-Tuning (SFT) to match foundation model behavior and Reinforcement Fine-Tuning (RFT) for 'deep thinking' by incorporating preference signals. 2. Highlighting the critical role of data quality through synthetic rewrites and LLM-as-a-Judge filtering for de-noising preference data. 3. Emphasizing the necessity of scaling effective batch sizes to over 1.5 million tokens using gradient accumulation to stabilize training and improve win rates against proprietary models. The partnership resulted in a 3.5x reduction in latency.

7/10/2026

This post details the engineering effort to optimize the MiniMax M3 sparse attention mechanism on NVIDIA Blackwell GPUs. It introduces a KV-stationary execution path, analyzes the performance trade-offs between Q-outer and KV-outer approaches, and describes specific optimizations for the attention store and load balancing. The resulting kernel achieves significant throughput improvements and speedups over existing baselines.

7/9/2026

This post details how Gumloop scaled its use of open-weight models by 7x in three weeks by integrating with Fireworks AI's inference platform. It highlights the successful migration of a critical internal agent from a proprietary model (Opus 4.8) to an open-weight model (GLM-5.2) without user-perceptible degradation, leading to significant cost savings (up to 72%). The post emphasizes Fireworks AI's reliability as a key factor in Gumloop's decision to adopt open-weight models for production workloads, enabling them to build production-ready AI agent workflows with improved economics.

7/8/2026

This post details the integration of NVIDIA Nemotron 3 Ultra with LangChain Deep Agents, emphasizing cost-per-task optimization for agentic workloads. It highlights Fireworks' inference stack, including NVIDIA Blackwell support and FireAttention kernels, for high-throughput and low-latency inference. The post also discusses the capability for enterprises to post-train Nemotron 3 Ultra on Fireworks for specialized intelligence, enabling ownership of AI models and competitive advantage.

7/8/2026

This post introduces Kimi K3 as a frontier model with a 2.8T parameter count and 1M context window, noting its pending weights and Fireworks listing. It also details the Kimi K2.7 Code model, highlighting its 1T parameter count and 256K context window for coding tasks. The post provides a comparative analysis of several open-source LLMs, including GLM 5.2, DeepSeek-V4-Pro, DeepSeek-V4-Flash, MiniMax M3, Qwen3.7 Plus, gpt-oss-120b, and Gemma 4 31B IT, discussing their technical specifications, benchmark performance, and suitability for different workloads. It also touches upon Fireworks' capabilities for deploying these models at scale.

6/30/2026

Introduced GLM 5.2 Fast, a new serving path for GLM 5.2 that achieves 2-3x higher inference throughput than the Standard path on shared serverless infrastructure. This is enabled by architectural optimizations including advanced Mixture-of-Experts (MoE) sharding strategies, sparse attention serving, and speculative decoding. The post details the trade-offs in MoE and attention sharding, emphasizing that different parallelism strategies are chosen independently based on workload characteristics. It also highlights the continued support for the full 1M-token context window, production-ready features like structured output modes and tool calling, and aggressive prompt-caching pricing to ensure speed does not compromise reliability or cost-effectiveness for agentic workloads.

6/26/2026

This post details how Fireworks provides the inference and rollout layer for large-scale distributed reinforcement learning, enabling Cursor to run RL across multiple global clusters. Key capabilities include cross-region model updates with significant transfer size optimization, minutes-level synchronization staleness, stable rollout fleets for MoE models, low-latency inference during training and evaluation, and reuse of production inference for RL sampling. This allows for accelerated RL cycles without dedicated inference infrastructure.

6/26/2026

This post details how Factory leveraged Fireworks AI to achieve 2-3x growth in open model usage within six months, driven by cost efficiency and vendor lock-in avoidance. Fireworks provides day-zero access to a broad range of open models with per-model fidelity, reliable performance, and managed infrastructure, enabling Factory to offer diverse model choices and optimize costs for its customers through features like Factory Router.

6/16/2026

Announced the availability of GLM 5.2 on the Fireworks inference platform, highlighting its performance on coding benchmarks and its 1M token context window for long-horizon agentic tasks. Detailed Fireworks' infrastructure advantages as a direct inference provider, emphasizing control, data privacy, and SLAs. Included customer testimonials and integration details for accessing GLM 5.2 via various APIs and tools.

6/12/2026

Announced the availability of Qwen 3.7 Plus on Fireworks' serverless infrastructure. Detailed its multimodal capabilities, reasoning modes, and context window. Highlighted Fireworks' role as a direct inference provider with specific performance and data handling commitments. Provided integration details for popular coding agents and API endpoints. Emphasized the distinction between Fireworks as an inference provider versus API routers.

6/11/2026

Launched Day-0 support for Kimi K2.7 Code, highlighting its efficiency gains (30% fewer reasoning tokens) and improved benchmark performance. Detailed the cost implications of reduced reasoning tokens for agentic workflows, emphasizing task completion cost over per-token pricing. Introduced and explained the Standard, Priority, and Fast serving tiers on Fireworks serverless infrastructure, detailing their use cases for agentic traffic.

6/11/2026

Added support for MiniMax M3, a new frontier model featuring long context (up to 500K tokens at launch, with 1M planned) and native multimodality. Highlighted the underlying MiniMax Sparse Attention (MSA) architecture and its performance benefits (e.g., 15x faster decoding at long context). Detailed M3's capabilities in coding and agentic tasks, and updated pricing information to include M3 alongside M2.7, with specific considerations for long-context pricing tiers.

6/1/2026

This post details Trilogy's validation of open-weight AI models for enterprise workloads using Fireworks AI. It highlights the challenges of scaling AI adoption, including escalating infrastructure costs and operational constraints like rate limits and usage caps. Trilogy selected Fireworks as a primary inference infrastructure layer for evaluation and early production workloads, enabling a shift from fragmented model experimentation to standardized evaluation and production-grade testing. Key results include standardized open-weight model usage, enabled billion-token scale agentic workflows (e.g., OpenSymphony), and reduced inference costs (~1/5th of proprietary systems). The post emphasizes Fireworks' role in providing production-grade inference infrastructure for open-weight models, allowing for rapid iteration, stable inference under high throughput, and a direct path from evaluation to production-grade testing.

5/22/2026

This post details the technical aspects of Fireworks AI's code assistance product, emphasizing context-aware code generation, inline fixes, and real-time autocomplete. It highlights low-latency performance through streaming completions with speculative decoding for sub-100ms response times, and scalable infrastructure leveraging GPU autoscaling and batching for cost-efficient handling of millions of concurrent requests. The post also quantifies improvements in developer output, including 2X faster code generation, 30% lower latency at scale, and 2.5X higher fix acceptance rates. It mentions the use of various models optimized for different tasks, such as Qwen3, Kimi K2 Instruct, and Deepseek models, and discusses fine-tuning capabilities for domain-specific accuracy. The technical depth is evident in the discussion of performance metrics and infrastructure scaling.

5/22/2026

This post expands on the Kimi K3 model deployment and API by detailing the broader platform capabilities for conversational AI. It highlights the ability to deploy fine-tuned models for reasoning, research, and writing, emphasizing accelerated insights, context maintenance, and smarter decision-making. Key technical aspects include enterprise-grade infrastructure with GPU autoscaling, high throughput, and predictable performance under load, enabling fast, scalable reasoning for multi-agent, multi-query workflows with sub-2s latency. It also mentions deep research automation, enterprise AI assistants, and real-time autocomplete. The post provides real-world impact metrics such as sub-2s latency, zero downtime, 50% higher GPU throughput, and successful scaling to 1.8M users in 24 hours, referencing a case study with Sentient.

5/22/2026

This post introduces Enterprise Agentic Systems, a new capability focused on enabling AI agents to perform multi-step workflows and interact with tools. Key technical contributions include: structured function calls for reliable tool invocation, support for multi-function and nested workflows, voice-to-action pipelines for speech-to-action conversion, fine-tuning capabilities with FireOptimizer for domain-specific language and workflows, and scalable infrastructure with GPU autoscaling for low-latency, high-concurrency inference. The post also quantifies performance improvements in terms of win rate, throughput, speed, and cost efficiency, and mentions specific model integrations like Qwen3 and Llama 3.1.

5/22/2026

This post details the application of Fireworks AI's platform to enterprise Retrieval-Augmented Generation (RAG) systems. It highlights the technical components involved in building RAG assistants, including fine-tuned embeddings, scalable re-ranking, multi-modal embeddings, long-context reasoning, and GPU autoscaling for low-latency, high-throughput inference. The post also mentions specific performance improvements like 4X faster processing, 4X cost efficiency, 5-7X higher order value, and sub-500ms transcription latency, and references a case study with DoorDash demonstrating high query throughput.

5/22/2026

This post announces the availability of the Kimi K3 model on Fireworks AI. It details the model's specifications (2.81T parameters, 1M-token context, visual understanding), deployment options (serverless with different tiers, on-demand), and supported functionalities (fine-tuning, function calling, image input). It also provides pricing information for serverless tiers and links to relevant documentation.

5/22/2026

This post introduces several new Kimi models (K2.7 Code, K2.6, K2 Instruct 0905, K2 Thinking, K2.5) that extend the Kimi family's capabilities. These models offer large context windows (up to 262144 tokens) and vision capabilities, indicating an expansion of the multimodal offerings and continued development in long-context reasoning and image understanding within the Kimi line.