Design an LLM Agent System That Automatically Resolves Jira Tickets and Opens Pull Requests
Design an LLM Agent System for Automated Jira Ticket Resolution and Pull Request Creation
You are tasked with designing an autonomous LLM-agent system capable of ingesting engineering Jira tickets (bug reports or small feature requests). The system should understand the relevant codebase, implement a fix, and automatically open a pull request (PR) for human review. The agent will operate on a real production code repository, making correctness, safety, and grounding its reasoning in the actual code paramount.
The design should focus on three key subsystems:
- Retrieval-Augmented Generation (RAG): This subsystem will enable the agent to access and utilize information from the codebase, the ticket itself, related tickets, documentation, and prior PRs to ensure its edits are grounded and avoid hallucinations.
- Tool Access via a Model Context Protocol (MCP)-style Integration Layer: This layer will provide a structured, permissioned interface for the agent to interact with the system, including reading files, performing searches, calling the Jira API, executing tests, and creating PRs.
- Sandboxed Execution Environment: This environment will allow the agent to safely edit code, run builds and tests, and iterate on fixes without endangering production systems or leaking sensitive information.
Design the end-to-end system, with a deep dive into these three subsystems: how the RAG index is built and queried, how the MCP tool layer is structured and secured, and how the sandbox is isolated and controlled.
Frame the design as a closed loop and anchor every decision to a potential failure mode.
Constraints & Assumptions:
- Repository Scale: A mid-to-large monorepo (10^4 – 10^6 files, tens of millions of lines of code) with multiple programming languages.
- Ticket Volume: Hundreds of eligible tickets per day; target latency is minutes to tens of minutes per ticket (asynchronous).
- Eligible Tickets: Scoped bugs and small feature requests that can be fixed with localized code changes (a few files).
- Human-in-the-Loop: The agent opens PRs for human review; it never merges. It may push follow-up commits based on review or CI feedback.
- Safety: The agent must not touch production infrastructure, exfiltrate secrets, and all code execution must be in an isolated sandbox. Builds and tests must pass before a PR is opened.
- Models: Assume access to a strong general-purpose LLM with tool-calling capabilities, a smaller/cheaper model, and an embedding model. Token budgets and per-ticket cost are considerations.
Similar LeetCode problem: Automate Git Commit Message Generation
AI interview prep
Core dilemma
Balancing the autonomy and potential for rapid code fixes with the critical need for safety, correctness, and preventing unintended consequences in a large, complex production codebase.
4 tradeoffs and 4 stress tests locked
Interview prep
Unlock the interviewer X-Ray rubric, tradeoff matrix, stress tests, clarifying questions, and key concepts.
Design an LLM Agent System That Automatically Resolves Jira Tickets and Opens Pull Requests
Design an LLM Agent System for Automated Jira Ticket Resolution and Pull Request Creation
You are tasked with designing an autonomous LLM-agent system capable of ingesting engineering Jira tickets (bug reports or small feature requests). The system should understand the relevant codebase, implement a fix, and automatically open a pull request (PR) for human review. The agent will operate on a real production code repository, making correctness, safety, and grounding its reasoning in the actual code paramount.
The design should focus on three key subsystems:
- Retrieval-Augmented Generation (RAG): This subsystem will enable the agent to access and utilize information from the codebase, the ticket itself, related tickets, documentation, and prior PRs to ensure its edits are grounded and avoid hallucinations.
- Tool Access via a Model Context Protocol (MCP)-style Integration Layer: This layer will provide a structured, permissioned interface for the agent to interact with the system, including reading files, performing searches, calling the Jira API, executing tests, and creating PRs.
- Sandboxed Execution Environment: This environment will allow the agent to safely edit code, run builds and tests, and iterate on fixes without endangering production systems or leaking sensitive information.
Design the end-to-end system, with a deep dive into these three subsystems: how the RAG index is built and queried, how the MCP tool layer is structured and secured, and how the sandbox is isolated and controlled.
Frame the design as a closed loop and anchor every decision to a potential failure mode.
Constraints & Assumptions:
- Repository Scale: A mid-to-large monorepo (10^4 – 10^6 files, tens of millions of lines of code) with multiple programming languages.
- Ticket Volume: Hundreds of eligible tickets per day; target latency is minutes to tens of minutes per ticket (asynchronous).
- Eligible Tickets: Scoped bugs and small feature requests that can be fixed with localized code changes (a few files).
- Human-in-the-Loop: The agent opens PRs for human review; it never merges. It may push follow-up commits based on review or CI feedback.
- Safety: The agent must not touch production infrastructure, exfiltrate secrets, and all code execution must be in an isolated sandbox. Builds and tests must pass before a PR is opened.
- Models: Assume access to a strong general-purpose LLM with tool-calling capabilities, a smaller/cheaper model, and an embedding model. Token budgets and per-ticket cost are considerations.
Similar LeetCode problem: Automate Git Commit Message Generation