TechPrufer
Get started
Baseten
Baseten logoBaseten · Feature trail

Agentic Code Transformation and Model Orchestration

16 posts · Jun 2024 – Sep 2026

What this trail covers

Baseten Chains is now generally available for production compound AI systems. Chains is an SDK for deploying performant compound AI systems to production, enabling AI builders to ship ultra-low-latency compound AI with unique hardware and autoscaling for each step. This eliminates performance bottlenecks and model orchestration headaches while keeping inference cost-efficient. Enhancements since beta include output streaming, binary IO support, subclassing for Chainlet reuse, Chains Watch for live code patching, a linter, more expressive logging, and UI improvements for managing environments.

All Baseten trails
  1. Introducing Baseten Hosted ToolsIntroduces Baseten Hosted Tools, an ecosystem for integrating external capabilities (like web search) into models and agents running on Baseten. Specifically details 'Baseten Grounded Inference' for web search, enabling models to access real-time web data. Highlights server-side tool execution to reduce latency and round trips, with live SSEs for observability. Demonstrates a curl example for enabling web search via a configuration change in Messages, Chat Completions, or Responses requests. Mentions integration with partner search providers (Exa, Keenable, Parallel, You.com) and future plans for code execution and browser automation. Benchmarks show a 15% decrease in end-to-end latency with server-side tool execution.
  2. Blaxel is joining Baseten to build the future of agentic infrastructureThis post announces the acquisition of Blaxel by Baseten, integrating Blaxel's agentic infrastructure primitives into Baseten's platform. Key contributions include: specialized microVM sandboxes with rapid suspend/resume (25ms) for state persistence at near-zero cost; Agent Drive, a distributed filesystem for durable and versioned artifact storage; and a rebuilt networking layer for secure and isolated agent communication. This integration aims to enable agents to act alongside inference models with low latency, forming a unified platform for model serving, training, and long-running agent execution.
  3. New MCP and skill for coding agents to use BasetenThis post introduces a backend MCP server and skill that enables coding agents to interact with the Baseten platform. The MCP server exposes the Baseten REST API to agents, offering discoverability and explicit annotations for read-only vs. mutating operations. Benchmarks show an average of 7.5% improvement in token usage, cost, and task completion time, with up to 57% improvement for operation-heavy tasks. The skill facilitates tasks such as deploying models, debugging deployments, tuning autoscaling, and managing day-to-day operations.
  4. Laguna S 2.1 goes Greek: a repository-scale game transformationThis post details the use of Laguna S 2.1, an agentic coding model, to perform a repository-scale transformation of the Hypersomnia game. It showcases Laguna's ability to understand an unfamiliar codebase, coordinate changes across interdependent files and tools, and continuously verify its work. The post highlights the orchestration of specialized models (Step 3.7 Flash for language adaptation, Krea 2 Turbo for asset generation and scaling via compositing) and demonstrates Laguna's creative judgment in model selection (choosing not to use Cosmos 3 Nano for video generation due to lack of role) and in making tradeoffs (prioritizing stable, playable builds over experimental, fully regenerated images). The transformation involved modifying 34 .cpp, 15 .h, 7 .hpp, and 8 JSON files, and compositing generated assets across roughly 945 sprites while preserving original dimensions and transparency masks.
  5. Building with NVIDIA Nemotron 3 Ultra and LangChain Deep Agents Code on BasetenThis post details the integration of NVIDIA Nemotron 3 Ultra with LangChain Deep Agents Code (dcode) on Baseten. It demonstrates how model-specific harness profiles, like the one for Nemotron 3 Ultra, can be applied to achieve high agent accuracy without requiring model fine-tuning or changes to serving infrastructure. The setup involves installing LangChain dcode with Baseten support and setting the Baseten API key. A simple agent example is provided to showcase how the agent reads workflow and control documents and generates readiness memos and risk tickets using file-reading and file-writing tools.
  7. Harnesses are everything. Here's how to optimize yours.This post introduces techniques for optimizing harness engineering for coding agents. It details three key areas: 1) keeping configuration files lean by using progressive disclosure for CLIs, skills, and MCP tools to manage context window usage; 2) structuring prompts using the R.P.I. framework (Research, Plan, Implement) to guide agents through problem-solving like staff engineers; and 3) employing subagents in parallel fan-out or pipeline patterns to maintain clean context in the main agent's window. These strategies aim to improve the usefulness and quality of AI-generated code by focusing on the harness's role in scaffolding intelligence.
  8. Secure your harness: how to run NVIDIA's NemoClaw with frontier open source modelsThis post details the integration of NVIDIA NemoClaw with Baseten's model serving infrastructure to provide a secure environment for running AI agents. It highlights NemoClaw's policy-based security, isolated sandboxes, and fine-grained controls, contrasting it with less secure alternatives. The post explains how Baseten's HIPAA and SOC 2 compliant model APIs, when paired with NemoClaw on a dedicated VPC, offer a secure solution for regulated industries by enabling sandboxed agents and private inference with zero cross-tenant exposure. It provides practical steps for registering Baseten as an inference provider within NemoClaw.
  9. Build AI agents in production (without writing code)This post details the integration of LangChain's Agent Builder with Baseten's GLM 4.7 LLM and Tavily for web search, enabling users to build AI agents with a no-code interface. It outlines the process of configuring Baseten as a custom OpenAI-compatible model API within Agent Builder, including specifying the base URL and API key. The guide demonstrates how to describe agent behavior in natural language and leverage pre-built templates or custom prompts to generate functional agents, highlighting Baseten's role as a performant inference backbone for agentic applications.
  10. Tool Calling in InferenceThis post details how inference providers directly influence tool calling success in AI agents. It breaks down the process into three key areas: pre-processing (chat template validation, model prompting), model execution (structured outputs, quantization techniques), and post-processing (parsing). The post emphasizes that while models are trained for tool calling, the inference provider's implementation at each stage is critical for reliability and quality. It also highlights the importance of benchmarking inference providers beyond model quality, focusing on aspects like schema validity, turn efficiency, and end-to-end latency for agentic workloads.
  11. Building AI agents, open code, and open source: A conversation with DaxThis post contributes to the agentic code transformation and model orchestration thread by discussing the philosophy and practical considerations behind building open-source AI coding tools (OpenCode) and curated inference services (Zen). It elaborates on the user experience challenges of integrating various LLMs, the importance of reliable inference providers, and the limitations of synthetic benchmarks, advocating for real-world evaluation. This provides context on the user needs and market landscape that drive the development of agentic platforms.
  12. From prompt to production: Baseten inference in your IDE with ClineThis post details the integration of Baseten's model serving infrastructure with Cline, an open-source AI coding assistant. It enables developers to use Baseten-hosted models (DeepSeek R1, Kimi K3, Qwen3 Coder) directly within their IDE for code generation and analysis. The post outlines the configuration steps for Cline to use Baseten as an API provider, including API key generation and model selection. It also provides a practical example of generating a playable Flappy Bird game using Cline's 'Act' mode with a Baseten-powered model, demonstrating the practical application of this integration for accelerating development workflows.
  13. Building reliable AI agentsThis post details the four-layer architecture of AI agents (cognitive, tool interaction, oversight, orchestration) and outlines best practices for building and deploying them in production. It emphasizes modular deployments with independent scaling per component, heterogeneous hardware support, fault isolation, latency-aware routing, and observability. The post highlights Baseten's capabilities in enabling modular deployment for agentic workloads and mentions integration with Patronus AI's Percival debugger for production agent failure analysis.
  14. Forward deployed engineering on the frontier of AIThis post introduces the concept and establishment of a Forward Deployed Engineering (FDE) function at Baseten. It details the strategic rationale for FDE, emphasizing its role in accelerating customer adoption of complex AI products by embedding engineers within customer organizations. The post outlines the unique characteristics of FDE, distinguishing it from consulting and solutions architecture, by highlighting its integration within the engineering organization and its direct contribution to product development and roadmap influence. It also discusses the challenges and strategies for recruiting, enabling, and scaling an FDE team, focusing on hiring for strong software engineering fundamentals and customer-facing skills. The FDE team's charter is to accelerate customer value and push the frontier of the Baseten product, particularly for mission-critical AI workloads and advanced model serving.
  15. Baseten Chains is now GA for production compound AI systemsThis post announces the general availability of Baseten Chains, an SDK for deploying compound AI systems. It details how Chains addresses challenges in model orchestration, latency, reliability, and cost for multi-model workflows. The core concept is 'Chainlets' which are individual subtasks that can run on custom hardware with independent autoscaling. Chainlets call each other directly, reducing latency. New features since beta include output streaming, binary IO support (NumPy arrays, raw bytes), subclassing for Chainlet reuse, Chains Watch for live code patching, a linter, improved logging, and UI enhancements for environment management.
  16. Building high-performance compound AI applications with MongoDB Atlas and BasetenThis post introduces the concept of compound AI and details how Baseten Chains, combined with MongoDB Atlas, can be used to build high-performance compound AI systems like RAG. It highlights the importance of appropriate hardware allocation (e.g., GPUs for LLM inference, CPUs for other tasks) and minimizing network overhead to eliminate bottlenecks. The post also discusses batching and throughput considerations in multi-stage AI workflows.
  17. Compound AI systems explainedThis post explains the concept of compound AI systems, which are workflows composed of multiple interacting AI models and processing steps. It details the benefits of this modular approach, including improved performance through specialized hardware allocation (e.g., CPUs for chunking, GPUs for transcription), increased flexibility by allowing individual components to be swapped or upgraded, and cost efficiency by using smaller, specialized models or task-specific hardware. The post also outlines the challenges in building, optimizing, and serving compound AI systems in production, such as coordination logic, I/O latency between models, and the need for infrastructure that mirrors the system's modularity. Baseten's 'Chains' SDK is presented as a framework for building and orchestrating these systems, enabling developers to combine business logic with ML models, customize hardware and scaling for distinct processing steps, and gain visibility into performance metrics across the entire workflow.
  18. Building multi-component AI workflows at scale with ChainsIntroduced Baseten Chains, a Python framework for building and orchestrating multi-component AI workflows. Chains are composed of 'Chainlets', modular services that can be linked for real-time inference. Key design goals include a delightful developer experience with local testability and type-checking, computational expressivity for flexible graph structures, and extensibility by reusing Truss features. Chains address pain points with Truss such as YAML configuration disconnects, code organization restrictions, and challenging error detection. The framework enables Chainlets to declare dependencies using `chains.depends()` and defines a `run_remote()` method as the public API for inter-Chainlet communication. Deployment involves a CLI command that generates Pydantic models for serialization and creates 'stubs' for remote instances, transforming Chainlets into FastAPI servers deployed on Baseten with autoscaling and dedicated hardware.
  19. Introducing Baseten ChainsIntroduced 'Chains', a new framework and SDK for building high-performance workflows with multiple models and components. Chains allows for modular workflow composition via 'Chainlets', which can be models or business logic. Each Chainlet can have independent GPU and CPU resource optimization and auto-scaling policies. The framework provides a better developer experience with features like auto-complete, static type-checking, and a simulated local execution environment. This has resulted in halved processing times and 6x improved GPU utilization in use cases like text-to-speech.