AI Agent Observability Interface (MCP)

3/4/2026

This post details the evolution of Datadog's MCP (Model Context Protocol) server, an observability interface for AI agents. It highlights key technical challenges and solutions: 1. Context Efficiency: Optimized data formats (CSV, YAML) and trimmed fields to reduce token usage by up to 5x. Introduced token-budget-based pagination instead of record-count pagination. 2. Querying Capabilities: Enabled agents to query data using SQL, allowing for efficient aggregation and filtering, reducing token usage and costs. 3. Tool Management: Strategies to manage tool count include flexible tools serving multiple use cases, opt-in toolsets for specialized needs, and layering for chaining tool calls. 4. Agent Guidance: Improved error messages for actionable feedback and integrated a `search_datadog_docs` tool for discoverable documentation. Tool results can now include contextual guidance.

9/23/2024

This post details the optimization of LLM usage for writing postmortems, focusing on cost, quality, and safety. It describes the integration of structured metadata from Datadog's Incident Management app with unstructured Slack discussions, fed into an ensemble of LLM models to generate postmortem drafts. Key challenges addressed include data quality and hallucinations, cost/speed/quality trade-offs, and trust/privacy. Solutions involved refining LLM instructions with both structured and unstructured data, establishing an experimentation framework with a custom API for data extraction and structuring, and implementing safeguarding mechanisms like prioritizing information, lowering model temperature, and adjusting LLM parameters to reduce hallucinations and improve reliability. The post also mentions an adjacent project for LLM-generated incident summaries that informed this work.