Fireworks AI Feature Trails

Feature trailActive

Frontier Model Training Infrastructure

The frontier-lab training infrastructure continues to evolve, now supporting post-training of models like GLM 5.2 into specialized agents. This involves advanced asynchronous reinforcement learning techniques, achieving significant improvements in agentic benchmarks with minimal regression on knowledge benchmarks and effectively flat cost per task. The platform's unified training and inference stacks ensure numeric alignment and batch invariance, offering end-to-end zero KLD across training and inference. This post specifically details the critical importance of numerical alignment between training and inference for RL, especially with MoE models, and how Fireworks addresses this through co-optimized trainer/rollout engines, router replay, and validation metrics like training-inference KL.

6 posts · 2026

Timeline

Feature trailActive

Multi-Region Deployment and Routing

The multi-region deployment capability allows users to run workloads across geographies with a single endpoint, optimizing for low latency, resilience, and operational ease. This system dynamically maps requests across the broadest possible eligible capacity pool while respecting accelerator types, quota, reliability tiers, compliance boundaries, and data residency constraints. The architecture has been rewired to allow a single deployment to draw compatible capacity from multiple regions, moving global scheduling off the inference hot path.

1 post · 2026

Timeline

Feature trailActive

Delta Compression for RL Weight Updates

The delta compression system for RL weight updates continues to evolve, with ARCv3 significantly reducing the size of BF16 weight updates sent from the trainer to rollout machines. This enables more practical cross-region RL training by keeping rollout fleets closer to the current policy and reducing the need for massive, co-located clusters. ARCv3 achieves this by exploiting the asymmetry in BF16 weight changes, encoding mantissa-only shifts efficiently and packing rarer exponent/sign changes separately. It also employs parallel general-purpose compression algorithms for optimal payload size.

1 post · 2026

Timeline

Feature trailActive

Model Inference Platform

The model inference platform continues to evolve beyond individual model capabilities, focusing on intelligent routing to optimize performance and cost. FireRouter, a new system, dynamically selects the best model for specific tasks within a larger workflow, demonstrating significant improvements in both accuracy and cost-efficiency compared to using a single, high-end model. This approach leverages a diverse pool of models, including open-weight options, to achieve superior collective intelligence. Ember-1, a specialized model derived from Kimi K3, offers comparable quality with significantly reduced token usage, setting a new Pareto frontier for cost-efficiency across various benchmarks and customer validation tests. This specialized model development, enabled by the Fireworks Serverless Training platform, allows for rapid iteration and deployment of optimized models.

11 posts · 2026

Timeline

Feature trailActive

Kimi K3 Model Deployment and API

Fireworks AI now offers the Kimi K3 model natively on its platform, enabling unified retrieval-to-response pipelines. This integration allows users to run embedding, retrieval, reranking, and generation on a single platform, reducing latency, cost, and security surface. The platform supports Kimi K3, optimized for various use cases including coding, writing, and multimodal analysis, with advanced features like Delta Attention for efficient large context handling. Fireworks provides US-hosted inference endpoints and offers tiered serverless options (Standard, Priority, Fast) for different reliability and performance needs. Fine-tuning with LoRA is supported for custom model deployment. On-demand deployments are available for dedicated GPU usage without rate limits.

31 posts · 2026

Timeline

Feature trailActive

Security Agent Economics

1 post · 2026

Timeline

Feature trailActive

Hybrid Worker-Advisor AI Architecture

The hybrid AI architecture combining open-source worker models with closed-source advisor models continues to be refined. This approach leverages the cost-efficiency of open-source models for task execution while utilizing frontier models for targeted review and quality assurance. Recent work has focused on applying this architecture to specific benchmarks like the Legal Agent Benchmark, demonstrating improved performance and cost-efficiency through system-level orchestration and post-training techniques. The platform's unified training and inference stacks facilitate seamless integration and iteration on these hybrid systems.

2 posts · 2026

Timeline

Feature trailActive

Billing System Migration

The billing system has been migrated from a postpaid to a prepaid model. Users can now purchase credits upfront, which are deducted as they use the platform. Auto-reload functionality is available to ensure continuous service. Contracted customers are unaffected. The system now supports automatic credit top-ups when a user-defined minimum balance is reached, and service pauses if credits are exhausted and auto reload is disabled. This post details the user-facing aspects of the migration and provides instructions for switching to prepaid billing.

2 posts · 2026

Timeline

Feature trailActive

Serverless Inference Tiers

Serverless 2.0 introduces three distinct serving paths for inference: Standard, Priority, and Fast. Standard is the default, cost-efficient option suitable for development and workloads tolerating occasional throttling. Priority offers stronger admission during congestion, shedding last under load to improve reliability for production workloads. Fast provides a high-throughput path for supported models, optimizing for faster generated-token throughput. This evolution aims to provide more control over reliability and throughput without requiring reserved capacity, allowing teams to scale from shared serverless to dedicated deployments more gradually. The platform also clarifies error codes, distinguishing between account-level rate limits (429 Too Many Requests) and fleet saturation (503 Service Overloaded), enabling more appropriate retry logic and alerting.

1 post · 2026

Timeline

Feature trailActive

Enterprise Agentic Systems

The platform continues to evolve its capabilities for building and deploying AI agents, with a new focus on optimizing the economics of security-focused agents. This includes evaluating and integrating models that demonstrate strong performance on adversarial security tasks while maintaining cost-efficiency, addressing the challenges of high-volume security workloads. The enterprise-grade AI agents are designed for multi-step workflow automation, featuring fast, reliable tool use with structured function calls, support for multi-function and nested workflows, and voice-to-action pipelines. They are optimized for low-latency, high-concurrency demands with GPU autoscaling and fine-tuning capabilities via FireOptimizer to align models with internal APIs and domain language. Enterprise governance features include full audit trails and more.

2 posts · 2026

Timeline