TechPrufer
Get started
Decagon
Decagon logoDecagon · Feature trail

Open Source LLM Productionization

3 posts · Nov 2025 – Jul 2026

What this trail covers

Decagon is increasingly leveraging open-source Large Language Models (LLMs) for production workloads, particularly for customer service AI agents. This strategy is driven by the need for low-latency, highly specialized models that can be customized for specific tasks. The company's experience suggests a maturity curve for AI use cases, where early-stage applications benefit from general-purpose frontier models, while mature, high-volume use cases transition to fine-tuned open-source models for improved performance and cost-efficiency. This post details the benefits of fine-tuning smaller models for specific tasks within a modular agent architecture, contrasting them with off-the-shelf models and outlining the use of Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL) for optimization. The approach emphasizes model independence and continuous refinement.

All Decagon trails
  1. Everyone is wrong about open source AI in the enterprise | DecagonThis post argues that the enterprise adoption of open-source AI models is driven by use case maturity, not just cost or customer demand. It explains Decagon's shift to ~90% open-source models due to the critical need for low-latency, fine-tuned models in production AI agent workloads like customer service. The post contrasts this with the broader enterprise trend of increasing spend on closed-source models, attributing it to the early stage of many new AI use cases. It posits that as use cases mature, the trend will reverse, with open-source models dominating production workloads.
  2. Introducing Decagon Labs | DecagonThis post introduces Decagon Labs, the research and agent orchestration arm of Decagon's engineering team. It highlights the company's strategic shift from relying on general-purpose foundation models to building their own network of specialized, post-trained models for enterprise customer experience (CX). This approach aims to achieve lower latency and higher accuracy tailored to specific functions like speech identification, workflow execution, and hallucination detection. The post emphasizes that Decagon Labs focuses on frontier AI research that directly translates to production impact, with a current emphasis on post-training techniques, agent architectures, and voice agents. It also announces four accompanying deep dives into related technical topics.
  3. AI agents are never done: The new build-vs-buy calculus | DecagonThis post elaborates on the continuous engineering effort required for AI agents beyond initial production deployment. It details the technical challenges and trade-offs involved in building and maintaining AI agents, including fine-tuning models, optimizing latency, ensuring reliability, evaluating providers, and implementing safety guardrails. It frames the build-vs-buy decision within the context of ongoing engineering investment, highlighting the costs associated with engineering time, performance gaps, operational risks, and infrastructure maintenance. The post emphasizes the importance of owning differentiating layers like workflows and domain logic while offloading infrastructure concerns. It critiques service-heavy vendor models and advocates for a product-first approach that enables direct user control and visibility into agent performance, referencing Decagon's specific product features that support continuous iteration and optimization.
  4. Engineering fast, performant AI agents through fine-tuning | DecagonThis post details Decagon's strategy of using fine-tuning to optimize smaller, specialized models for specific functions within their AI agent architecture, aiming for both low latency and high accuracy. It contrasts this with the limitations of off-the-shelf general-purpose LLMs and outlines the use of Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL) as complementary techniques for model adaptation. The post highlights the trade-offs of RL (instability, reward model complexity, computational demand) versus SFT (efficiency, predictability) and describes a typical workflow of starting with SFT and layering RL. It also emphasizes the benefits of a modular, model-agnostic architecture that allows for continuous evaluation and refinement of individual components.