TechPrufer
Get started
Datadog
Datadog logoDatadog · Feature trail

AI Agent Observability Interface (MCP)

2 posts · Sep 2024 – Mar 2026

What this trail covers

Datadog's MCP (Model Context Protocol) server has evolved significantly to provide an observability interface specifically designed for AI agents. Initial versions were simple API wrappers, but the system has been re-architected to address agent constraints like context window efficiency, data retrieval limitations, and tool management. Key developments include optimizing data formats (CSV, YAML over JSON), implementing token-budget-based pagination, introducing query languages (SQL) for data analysis, and improving error messages and documentation discoverability. The platform supports flexible tools, toolsets, and layering for specialized needs, while also exploring the integration of agent skills. The goal is to make observability ergonomic for agents, reducing token usage and improving accuracy.

All Datadog trails
  1. Designing MCP tools for agents: Lessons from building Datadog’s MCP server | DatadogThis post details the evolution of Datadog's MCP server to improve observability for AI agents. Key technical contributions include: optimizing data formats from JSON to CSV/YAML for token efficiency (achieving up to 5x more records per token), implementing token-budget-based pagination instead of record-count pagination, introducing SQL as a query language for agents to retrieve specific data rather than raw logs (reducing costs by ~40% in eval scenarios), designing flexible tools and toolsets to manage tool count, exploring layering for specialized functionality (trading latency for reduced context), improving error messages for agent recoverability, and making documentation discoverable via a RAG-powered search tool. The post also discusses the trade-offs between specialized agents and general MCP servers.
  2. How we optimized LLM use for cost, quality, and safety to facilitate writing postmortems | DatadogThis post details the optimization of LLM usage for writing postmortems, focusing on cost, quality, and safety. It describes the integration of structured metadata from Datadog's Incident Management app with unstructured Slack discussions, fed into an ensemble of LLM models to generate postmortem drafts. Key challenges addressed include data quality and hallucinations, cost/speed/quality trade-offs, and trust/privacy. Solutions involved refining LLM instructions with both structured and unstructured data, establishing an experimentation framework with a custom API for data extraction and structuring, and implementing safeguarding mechanisms like prioritizing information, lowering model temperature, and adjusting LLM parameters to reduce hallucinations and improve reliability. The post also mentions an adjacent project for LLM-generated incident summaries that informed this work.