Model Inference Platform · Fireworks AI Feature Trail

Sep 2026Introducing Ember-1Introduced Ember-1, a specialized model derived from Kimi K3, which achieves comparable quality with 40% fewer tokens by optimizing reasoning efficiency. This was accomplished through extensive research and over 50 training experiments using new training algorithms on the Fireworks Serverless Training platform. Ember-1 was evaluated on industry benchmarks, including the Specialized Intelligence Index (SII) and customer A/B tests, demonstrating significant cost savings (35-50% fewer tokens) without sacrificing accuracy. Internal validation showed that developers did not notice the model switch, indicating a seamless integration. This release marks the beginning of a series of specialized models from Fireworks Research.

Sep 2026The frontier isn’t a model. It’s a router.This post introduces FireRouter, a system designed to dynamically select the optimal model for individual tasks within a larger AI workflow, moving beyond the concept of a single 'frontier' model. It presents empirical evidence from the DeepSWE benchmark, showing that a router can achieve 97.6% task completion at $1.88 per task by intelligently selecting from 18 models, a significant improvement over using a single best model like GPT-6 Astra (74.1% at $6.52). The analysis highlights that a few complementary models are sufficient to capture most of the collective capability, and that the challenge lies in predictable model selection. FireRouter is presented as a cache-aware system that routes at the task level, aiming to improve capability curves rather than just cost. Production traffic analysis shows a 53% cost reduction compared to using Opus 5 alone.

Sep 2026Phylo brings frontier AI to more scientists with open models on FireworksThis post details Phylo's migration from proprietary models to serving open-weight frontier models on Fireworks AI for their agentic biology environment (Biomni Lab). Key technical contributions include: leveraging Fireworks' serverless fast path endpoints for day-zero access to SOTA open models, achieving a 24-hour cycle from model release to production; optimizing for long-horizon agentic inference (hours to days, hundreds of tool calls) with a focus on cost efficiency; and demonstrating significant cost reduction (60% lower cost) and performance improvements (halved time to first token) by shifting the majority of traffic to open models served via Fireworks. The post highlights the importance of model-agnostic design, automated evaluation pipelines (BiomniBench), and direct engineering collaboration for rapid deployment and production stability.

Sep 2026DeepSeek-V4.1-Flash on Fireworks: Astra-level DeepSWE at 1/15th the costThis post details the integration of DeepSeek-V4.1-Flash onto the Fireworks platform, emphasizing its cost-performance benefits for autonomous software engineering. It introduces the model's split activation architecture (552B MoE with 8B input, 16B output active parameters) and KV cache optimizations (4x HBM reduction, 8x SSD overhead reduction) that significantly reduce token spend, especially for input-heavy agentic workflows. Benchmarks on DeepSWE and Terminal-Bench 2.1 show comparable quality to higher-cost models like GPT-6 Astra at a fraction of the price. It also highlights DeepSeek-V4.1-Flash's limitations on complex reasoning tasks (HLE) and suggests multi-model routing as a strategy to achieve superior performance.

Aug 2026DeepSeek V4 Pro: Tops SWE-Bench & Cuts Cost per Task by 3x vs. Fable 5This post introduces DeepSeek V4 Pro 0813 to the Fireworks AI platform, detailing its performance on SWE-Bench Verified, LiveCodeBench v6, Aider Polyglot v1, and Terminal-Bench 2.1. It highlights DeepSeek V4 Pro's superior accuracy on SWE-Bench and LiveCodeBench compared to Kimi K3 and Fable 5, and its significant cost advantage over Fable 5. The post also discusses oracle routing strategies between DeepSeek V4 Pro and Fable 5, demonstrating a substantial improvement in accuracy and cost reduction. It notes DeepSeek V4 Pro's limitations in multi-language breadth, particularly in Java, and emphasizes the benefit of using specialized models with dynamic routing. Training support for DeepSeek V4 Pro is now available, including SFT, DPO, and RFT, with a focus on co-optimized training and serving infrastructure. The post also provides insights into DeepSeek's native harness (dsh) and its implications for tool calling consistency.

Aug 2026Fireworks AIThis post introduces DeepSeek V4 Pro 0813 to the Fireworks AI inference platform, detailing its exceptional performance on the CyberGym security benchmark. It highlights zero refusals and native tool calls across 840 adversarial security tasks, achieving a 53.7% reward rate at $1.343 per task and $2.502 per reward. The analysis shows DeepSeek V4 Pro outperforms GPT-5.5 in both accuracy and cost-efficiency for security agent workloads, costing a quarter as much per solved vulnerability. It also details that DeepSeek V4 Pro fails at the setup step (generating the initial exploit) rather than the hard part (fixing code), suggesting prompt-and-scaffolding improvements.

Aug 2026Can open models carry readable silent signals before they speak? Reproducing J-Lens Readouts on Kimi K3 & Qwen3.5-9BThis post details the application of the J-Lens technique to reproduce 'silent signals' in open models Kimi K3 and Qwen3.5-9B. It describes the experimental setup, including paired copy experiments and calibration passages, to demonstrate that internal model states can reveal intended vocabulary before it is generated. The analysis highlights the success in identifying concept families (e.g., arithmetic, citrus) and specific tokens in the model's internal states, even when the visible output is identical. It also discusses a limitation encountered due to tokenizer differences, which can affect the accuracy of signal recovery.

Aug 2026Fireworks AIThis post introduces Muse Glimmer, a 30B dense model from Meta, to the Fireworks AI platform. It details Muse Glimmer's architecture, highlighting its 52 transformer layers, grouped-query attention, SwiGLU feed-forward layers, and a ~1.8B perception encoder for native image understanding. Key architectural features for agentic workloads include sliding-window attention over 2,048 tokens on most layers with global attention every fourth layer, and two KV heads to minimize KV cache size, enabling economical serving of long-context agents at high concurrency. The post also mentions DFlash speculative decoding for lower-latency generation. It provides benchmark comparisons against Gemma 4 31B and Qwen 3.6 27B, emphasizing Muse Glimmer's strengths in tool orchestration and sustained multi-turn work. Fireworks AI's contribution includes correcting the model's generation config and wiring reasoning-effort control into the system prompt, offering serverless and on-demand deployments optimized for bursty agent traffic.

Jul 2026Fine-Tune Your Own Embedding Model from an LLM — for the Price of a CoffeeThis post details the process and benefits of fine-tuning a general-purpose LLM (Qwen3-Embedding-8B) into a domain-specific embedding model using contrastive learning with in-batch negatives and the InfoNCE loss. It presents experimental results showing significant improvements in retrieval tasks (e.g., +36% nDCG@10 on legal citation retrieval) compared to base models. The post also outlines three Fireworks training modes (full-parameter updates, LoRA adapters) and provides a code example for training on private data, highlighting the cost-effectiveness of this approach.

Jul 2026Fireworks AIIntroduced Fireworks Nexus, a product designed to help engineering organizations manage and optimize their AI spend. This includes enterprise controls and cost observability, workflow continuity via FireConnect (a one-line install for mapping models and API compatibility), and intelligent traffic management with FireRouter. FireRouter dynamically routes requests based on complexity, directing routine tasks to cost-effective open-weight models (like Kimi-K3 and GLM 5.2) and complex tasks to existing frontier providers. The post also references external evaluations by Faros and Arize demonstrating cost savings and comparable or better performance with open-weight models when intelligently routed.

Jul 2026Trilogy’s Playbook for Open-Weight Cybersecurity with Kimi K3This post details the integration of Kimi K3 with Fireworks' OpenAI-compatible API, enabling its use in cybersecurity workflows. It highlights how Fireworks provides the managed inference layer for Kimi K3, allowing teams to leverage open-weight models for tasks like repository auditing, vulnerability triage, and incident timeline preservation. The post emphasizes the separation of deterministic tooling from model reasoning, with Fireworks handling the model serving infrastructure. It also provides setup instructions for integrating Kimi K3 on Fireworks into a cybersecurity playbook.

Jul 2026Kimi K3 is competitive with Fable; Kimi K3 + Fable is SoTA.This post introduces Kimi K3 as a competitive open model, demonstrating its ability to achieve 93% accuracy when routed with Fable 5, at up to 50x lower cost. The analysis highlights K3's strengths in specific task domains (e.g., symbolic math, dev tooling, terminal tasks) and its cost-effectiveness across various work types, advocating for a routing-based approach over single-model selection. It provides detailed benchmarks across SWE, Terminal, Algorithmic, Multi-Language, and Legal tasks, comparing K3 and Fable 5 on accuracy and cost, and illustrating the benefits of an 'oracle routing' strategy.

Jul 2026Heidi x Fireworks: Bridging the Gap in Frontier Model PerformanceThis post details the technical approach taken by Heidi Health in partnership with Fireworks AI to achieve superior performance from open-source models compared to proprietary frontier models. Key technical contributions include: 1. A rigorous data filtering strategy for Supervised Fine-Tuning (SFT) and Direct Preference Optimization (DPO), involving synthetic rewrites and LLM-as-a-Judge for quality assurance and de-noising. 2. The use of large effective batch sizes (up to 1.5 million tokens) enabled by gradient accumulation on the Fireworks AI platform to stabilize training and overcome residual noise in preference data. The post quantifies performance improvements, showing a drop in latency from 25 seconds to 7 seconds and a win rate improvement over proprietary models from 48.0% to 51.3% by increasing batch size.

Jul 2026Optimizing MiniMax M3 Sparse Attention on NVIDIA BlackwellThis post details the engineering effort to optimize the MiniMax M3 sparse attention kernel on NVIDIA Blackwell hardware. Key contributions include: implementing a KV-stationary execution path to improve memory access patterns and reduce redundant loads; developing a warp-specialized sparse attention main kernel with optimized load (TMA, cp.async gather) and store (partial-O emission) paths; and designing a four-stage pipeline (index build, scheduler, sparse attention main kernel, combine) to manage the workflow. The optimizations focus on reducing gathered-load/scattered-store memory traffic and improving load balancing, resulting in significant performance gains over baseline kernels.

Jul 2026How Gumloop Scaled Open-Weight Model Usage 7x in 3 Weeks with Fireworks AIThis post details Gumloop's successful strategy for scaling the usage of open-weight models by 7x in three weeks, driven by their partnership with Fireworks AI for inference. Key technical contributions include: 1. Internal validation of open-weight models (specifically GLM-5.2) against frontier models (Opus 4.8) for production agent workloads, with no noticeable difference in user experience or output consistency. 2. Optimization of Gumloop's agent harness to effectively utilize open-weight models for tasks like information extraction, classification, routing, and summarization. 3. Achieving significant cost savings (up to 72%) by migrating a critical internal agent from Opus 4.8 to GLM-5.2, highlighting the economic benefits of open-weight models when paired with reliable inference infrastructure. 4. Gumloop's selection of Fireworks AI as their default inference provider due to its demonstrated reliability and fast response times, which provided the confidence to recommend open-weight models for production.

Jul 2026Open, frontier, and yours: LangChain Deep Agents on NVIDIA Nemotron 3 Ultra, running on FireworksThis post details the integration of NVIDIA Nemotron 3 Ultra with LangChain's Deep Agents harness on the Fireworks inference platform. It highlights the cost-per-task metric for agentic workloads, emphasizing the economic advantage of open models like Nemotron 3 Ultra when served on optimized infrastructure. The post also details Fireworks' custom FireAttention kernels and disaggregated engine architecture, designed to maximize throughput for large models and long agent runs, leveraging NVIDIA Blackwell and Blackwell Ultra. It further elaborates on the ability to post-train models like Nemotron 3 Ultra on the same platform for specialized intelligence, using techniques like supervised fine-tuning and direct preference optimization.

Jul 2026Best Open Source LLMs in 2026: We Reviewed 7 ModelsThis post provides a comprehensive review and comparison of several leading open-source LLMs (GLM 5.2, Kimi K3, Kimi K2.7 Code, DeepSeek-V4-Pro, DeepSeek-V4-Flash, MiniMax M3, Qwen3.7 Plus, gpt-oss-120b, Gemma 4 31B IT) based on benchmark performance, context window, modality support, and licensing. It highlights GLM 5.2 and Kimi K3 as strong contenders for general reasoning, coding, and long-context agents, and discusses how Fireworks AI's platform supports the deployment and scaling of these models, including performance at scale and deployment options.

Jun 2026GLM 5.2 Fast is live on FireworksIntroduced GLM 5.2 Fast, a new inference path that achieves 2-3x higher generated-token throughput compared to the Standard path on shared serverless infrastructure. This optimization is achieved through architectural improvements in the Mixture-of-Experts (MoE) MLP stack and the DeepSeek Sparse Attention stack, including strategies for expert sharding, tensor parallelism, and data-parallel attention. The post details the trade-offs involved in these parallelism strategies, such as memory/communication trade-offs for MoE and the serving catch for MLA attention. It also highlights the use of speculative decoding and the separation of prefill from decode to enhance speed without compromising quality or reliability, particularly for agentic workloads with long context and tool calls. The prompt caching mechanism is further emphasized as a key cost-saving and practicality enabler for agentic workflows.

Jun 2026Fireworks AIThis post details how Fireworks AI's inference platform enables large-scale, distributed reinforcement learning for coding models by providing the infrastructure for rollout and inference across multiple clusters. It highlights the use of compressed weight synchronization for efficient model updates across regions, the reuse of production inference for RL sampling and rollout execution, and the resulting acceleration of RL training cycles. The platform's capabilities allow Cursor to focus on model behavior rather than low-level systems scaling.

Jun 2026How Factory Grew Open Model Usage 2-3x in Six Months on FireworksThis post details how Factory achieved 2-3x growth in open model usage and significant cost reductions (6-20% of frontier model costs) over six months by leveraging Fireworks AI's platform. Key technical contributions include: Fireworks' day-zero availability of open models, per-model fidelity (handling diverse API shapes, reasoning traces, tool schemas), and reliable performance across various model families. This enabled Factory to implement their 'model independence' strategy, allowing Droids to run on any credible coding model and absorb the complexities of diverse model behaviors. The post quantifies cost savings per task and increased throughput per dollar, illustrating the economic viability of open models at scale when delivered reliably.

Jun 2026GLM 5.2 is live on Fireworks inference, day zero.This post announces the day-zero availability of GLM 5.2 on the Fireworks inference platform. It highlights the model's strengths in coding and its 1M-token context window for long-horizon agentic tasks. The post emphasizes the importance of running models on Fireworks' own infrastructure for controlled serving, zero-data retention, and SLAs, contrasting this with router-based API access. It also details how the Fireworks inference stack supports the reliability of large context windows under engineering pressure, and provides integration guides for using GLM 5.2 with popular agent frameworks and APIs.

Jun 2026Qwen 3.7 Plus is now live on FireworksThis post announces the direct hosting and serving of Qwen 3.7 Plus on Fireworks infrastructure, positioning Fireworks as an inference provider rather than just an API router. It highlights the model's agentic capabilities, multimodal input, and a 262k context window. Key technical details include the selectable reasoning modes (`reasoning_content` and `reasoning_history` parameters) for agent loops, serverless pricing, and API compatibility (OpenAI and Anthropic). The post emphasizes the benefits of direct hosting, such as controlled latency, data governance (zero data retention), and reliability SLAs, contrasting it with API routers. It also notes the licensing distinction for Qwen 3.7 Plus compared to open-weight Qwen models.

Jun 2026Kimi K2.7 Code on Fireworks: Better Agents, Lower Cost per Task, Available Day-0 | Fireworks AIThis post announces Day-0 support for Moonshot's Kimi K2.7 Code model on Fireworks. The key technical contribution is the model's improved reasoning efficiency, using ~30% fewer reasoning tokens than K2.6 while achieving higher scores on coding benchmarks. The post explains the compounding cost implications of verbose reasoning in agentic workflows and highlights that the primary metric for evaluating agent performance is now cost per completed task, rather than just per-token pricing. It also details the three serving options (Standard, Priority, Fast) available on Fireworks serverless infrastructure for handling bursty agentic traffic, emphasizing the trade-offs between reliability and speed.

Jun 2026MiniMax M3 is live: long context + native multimodality at 1/20th the priceIntroduced MiniMax M3, an open-weight model with native multimodal understanding (images and video) and a 1M-token context window. Details the MiniMax Sparse Attention (MSA) architecture, which scales sub-quadratically by implementing a pre-filtering stage that partitions KV caches into blocks, achieving up to 15x faster decoding at long context compared to M2.7 and a 95% drop in per-token compute at 1M tokens. Highlights M3's performance on coding and agentic benchmarks, surpassing other open-source models and some closed-source models. Discusses pricing for M3 on Fireworks, noting it is 2x the price of M2.7 for serverless inference, with a higher long-context rate for inputs >512K tokens.

Jun 2026NVIDIA Nemotron 3 Ultra is live on Fireworks, day zeroThis post announces the day-zero availability of NVIDIA Nemotron 3 Ultra on the Fireworks inference platform. It highlights the model's suitability for long-running autonomous agents due to its optimization for task completion cost and time, rather than just prompt response. The post details the platform's proprietary optimizations, such as FireAttention custom kernels, which deliver up to 4x higher throughput for agentic tasks. It also emphasizes the ability to train and deploy Nemotron 3 Ultra on the same platform using SFT and DPO, ensuring seamless transition from training to production with optimized serving infrastructure. On-demand deployments with dedicated GPUs are also promoted for predictable performance and cost-effectiveness.

May 2026Trilogy Validates Open-Weight AI Models for Enterprise Workloads with Fireworks AIThis post details Trilogy's adoption of Fireworks AI as their primary inference infrastructure for open-weight models. It highlights the challenges of scaling proprietary AI models due to cost and operational constraints (rate limits, usage caps). Trilogy selected Fireworks for its ability to deploy open-weight models without infrastructure overhead, rapid switching across model variants, stable inference under high token throughput, and a direct path from evaluation to production. The post specifically mentions the use of Fireworks to power OpenSymphony, an internal work orchestration system for multi-agent execution, enabling billion-token scale agentic workflows and achieving approximately 1/5 the cost of proprietary systems for comparable workloads.

May 2026Serverless 2.0: Three Ways to Run Inference, One APIIntroduced Serverless 2.0 with three distinct serving paths: Standard (default, cost-efficient, sheds first under load), Priority (stronger admission during congestion, sheds last), and Fast (high-throughput serving path for supported models, ~2x Standard throughput). Differentiated HTTP 429 (rate limit) from 503 Service Overloaded (fleet saturation) and 503 Service Unavailable (platform issue). Priority is activated via `service_tier: "priority"` and Fast via specific model router IDs (e.g., `accounts/fireworks/routers/kimi-k2p6-turbo`).

May 2026Accelerate Developer Output Across Your EnterpriseThis post details the technical capabilities of the Fireworks AI platform for accelerating developer output through context-aware code generation, inline fixes, and real-time autocomplete. It highlights low-latency streaming performance (sub-100ms response times), scalable infrastructure with GPU autoscaling and batching, and the ability to fine-tune models on internal codebases for idiomatic, architecture-aware output. The post also quantifies impact with metrics like 2X faster code generation, 30% lower latency at scale, and 2.5X higher fix acceptance rates, supported by case studies from Cursor and Sourcegraph.

May 2026Enterprise Agentic Systems for Multi-Step Workflow AutomationThis post introduces Fireworks Agentic Systems, a new capability focused on enabling enterprise-grade AI agents that can act by executing workflows, handling complex tasks, and scaling across an enterprise. Key technical contributions include: fast, reliable tool use via structured function calls for multi-step workflows; support for multi-function and nested workflows with schema consistency; voice-to-action pipelines converting speech to structured, real-time actions; integration with FireOptimizer for fine-tuning models to internal APIs, workflows, and domain language; scalable infrastructure with GPU autoscaling to support millions of tool calls with consistent low latency; and enterprise governance features like full audit trails, monitoring, and controls for compliance and reliability. It also highlights recommended models optimized for reasoning, planning, and tool orchestration, and showcases real-world impact metrics and a case study with Notion.

May 2026Fireworks AIThis post introduces and quantifies the 'Agent Execution Tax,' defining it as the ratio of wasted inference to productive inference in agentic AI systems. It presents a benchmark of 720 browser automation tasks across three LLMs (GLM-5, MiniMax M2.5, Kimi K2.5) to measure structured output reliability and step efficiency. Key findings include that Kimi K2.5 had zero parse retries, MiniMax M2.5 offered the best value per successful task due to low retry rates, and GLM-5 provided the highest accuracy but at a higher cost. The post argues that inference infrastructure significantly impacts execution tax through structured output consistency and latency predictability, and proposes 'Reliability-Adjusted Accuracy' as a critical procurement metric. It details the benchmark setup, models used, and a per-site analysis of success and failure modes.

May 2026Innovative Solutions Rebuilds Enterprise Services Delivery with Fireworks AIThis post details how Innovative Solutions rebuilt their enterprise services delivery by integrating their DarcyIQ platform with Fireworks AI's inference layer. Key technical contributions include: enabling predictable scaling of multi-agent AI workloads (4-10B tokens/month), transforming AI inference from a linear cost center into a predictable scaling layer, reducing model integration overhead and operational complexity, stabilizing multi-model execution, and compressing contract cycles from 30-45 days to ~3 days through real-time contract and scope generation. The post highlights the architectural shift towards agentic delivery systems and the economic benefits of stable, high-throughput inference.

UndatedDeepSeek-V4-Pro-0813 API & Playground | Fireworks AIIntroduced DeepSeek-V4-Pro-0813 to the model inference platform. This model features enhanced agentic capabilities and performance improvements, especially in production. It is built on the DeepSeek-V4-Pro structure with a DSpark speculative decoding module. The platform now supports this model via serverless API (pay-per-token) and on-demand deployment on dedicated GPUs. Key specifications include 1.6T parameters, Mixture-of-Experts architecture, and a context length of 1040k tokens. Function calling is supported, but embeddings and image input are not.

UndatedGLM-5.3 API & Playground | Fireworks AIThis post announces the availability of GLM-5.3 on the Fireworks AI inference platform. It highlights that GLM-5.3 is a post-trained version of GLM-5.2, offering significant improvements in complex coding and long-horizon tasks. The post details its availability via serverless API (pay-per-token) and on-demand deployment on dedicated GPUs, emphasizing high reliability and no rate limits. Pricing for serverless usage is provided. Technical specifications include 743B parameters and a 1040k token context length. Fine-tuning is noted as not supported for this specific model version.

What this trail covers