AI Agent Platform for Engineering

4 posts · Sep 2024 – Jun 2026

What this trail covers

Dropbox is evolving its engineering practices beyond AI code generation tools to AI agent platforms like Nova. Nova allows engineers to describe tasks in natural language, and AI agents execute these tasks within a controlled environment, interacting with the codebase, running tests, and iterating on failures. This platform aims to accelerate the broader software development lifecycle by handling scoped tasks, including feature development, migrations, test remediation, and bug investigation. This post details the platform's architecture, its use in CI failure remediation, dependency upgrades, and emerging workflows, highlighting the importance of validation, context handling, and integration with existing tools like Bazel and RenovateBot.

All Dropbox trails

Jun 2026How we used DSPy to turn AI evaluations into better responses in Dash chatThis post details the application of DSPy, an open-source framework for optimizing AI systems, to improve the quality of AI evaluations and subsequently enhance the responses of the Dash chat agent. It describes a two-stage process: first, calibrating LLM judges against human-labeled examples using DSPy's optimization algorithms (GEPA, MIPROv2) to align judge scores with human judgment; and second, using these improved judges to optimize the chat agent's system prompt through an automated, evaluation-driven feedback loop. This loop replays candidate prompts on historical chats, scores the outputs, and uses the feedback to propose prompt updates, focusing on concrete failure modes like wrong context selection and incomplete answers.
May 2026Beyond code generation: rethinking engineering productivity in the age of AI agentsThis post details the evolution of Dropbox's approach to AI in engineering, moving from AI coding assistants (copilots) to AI agent platforms like Nova. It explains how Nova enables engineers to delegate scoped tasks to AI agents, which can inspect code, edit files, run tests, and iterate on failures. The post highlights that the value of Nova comes from the surrounding systems (codebase context, engineering practices, safe execution, workflow integration, human review) rather than just the models. It also discusses the shift in measuring engineering productivity from code output (e.g., PR throughput) to broader system outcomes, including workflow adoption, production output, and customer impact, while emphasizing the importance of quality and trust. The post also touches on the need for engineers to adapt their workflows and the importance of enablement alongside tooling for successful agentic adoption.
May 2026Introducing Nova, our internal platform for coding agentsThis post introduces Nova, an internal platform for running coding agents within Dropbox's development environment. It details the platform's architecture, which supports interactive sessions and asynchronous workflows, and its integration with existing tools like Bazel for validation and RenovateBot for dependency upgrades. Key use cases highlighted include flaky test remediation and migration efforts, emphasizing the importance of a validation loop and context handling across multiple steps. The platform's design prioritizes operating within Dropbox's specific infrastructure and validation paths, moving beyond single-purpose solutions to a shared system for AI-assisted engineering tasks.
Mar 2026How we optimized Dash's relevance judge with DSPyThis post details the optimization of Dropbox Dash's relevance judge using the DSPy framework. The primary technical contribution is the application of DSPy's GEPA optimizer to adapt a relevance judging model (originally tuned for a proprietary model) to a more cost-effective open-weight model (gpt-oss-120b). This involved defining a clear objective based on human agreement (NMSE) and structural reliability, and using DSPy's reflection loop to generate structured feedback for iterative prompt refinement. The post also outlines the process of measuring human agreement and the challenges of prompt brittleness when migrating between models.
Nov 2025How Dash uses context engineering for smarter AIThis post details how Dropbox's Dash product is evolving from a traditional RAG pipeline to an agentic AI system by focusing on 'context engineering'. Key technical contributions include: 1. **Retrieval Consolidation:** Replacing multiple retrieval tools with a single, purpose-built tool backed by the Dash universal search index to simplify the model's decision space. 2. **Context Filtering with Knowledge Graph:** Building a unified index and layering a knowledge graph to connect people, activity, and content, enabling relevance-based ranking of retrieved information. 3. **Specialized Agents:** Moving complex tools like query construction into dedicated agents with focused prompts to allow the main planning agent to focus on higher-level execution. The post also highlights the use of the Model Context Protocol (MCP) and the development of a Dash MCP server for integrating with other apps.
Advertisement
Oct 2025A practical blueprint for evaluating conversational AI at scaleThis post details the development of a standardized evaluation process for conversational AI systems, treating every experiment like production code. It covers curating datasets (public and internal), defining actionable metrics using LLMs as judges, and setting up an evaluation platform (Braintrust) to manage datasets, scorers, experiments, and automation. The goal is to ensure accuracy and reliability in AI outputs, extending to various modalities beyond text.
Sep 2024How we use Lakera Guard to secure our LLMsThis post details the integration of Lakera Guard to secure LLMs used in Dropbox's AI-powered features. It describes the evaluation criteria for LLM security solutions, focusing on in-house deployability, low latency, support for long context lengths, and confidence scores. The testing methodology involved using Garak for vulnerability scanning and measuring net security improvement against LLM mitigations. The integration architecture uses LangChain and Lakera Guard as an internal Docker service, with RPC calls to Lakera's prompt injection and content moderation APIs. Key technical achievements include a 7x latency improvement for prompts over 8,000 characters and collaboration with Lakera on detecting novel LLM vulnerabilities. The post also outlines plans for broader integration across Dropbox products.