Building an agentic harness that outlasts the model (2026) - Shopify
Jul 29, 2026

Summary
This post details the development of an "agentic harness" named Dispatch for automated security vulnerability discovery and remediation. It outlines the multi-agent orchestration workflow, including stages for test bootstrap, architecture documentation, file cataloging, partitioning, hunting, verification, post-processing, reporting, and remediation. Key technical contributions include strategies for code partitioning to manage costs and improve accuracy, the development of a test oracle for web vulnerabilities, and lessons learned in producing consistent and reliable results from AI models. The post emphasizes the importance of the harness outlasting the models themselves.
More from the AI Assistant (Sidekick) Capabilities and Training trail
- Gisting: Compressing LLM Agent context to ↑ throughput and ↓ cost (2026) - Shopify
- Sidekick's continual learning loop (2026) - Shopify
- Teaching Sidekick to say no: automated data curation with LLM judge consensus (2026) - Shopify
- Building a Magic Mirror: AI retail experiences with Remix (2026) - Shopify
- Building production-ready agentic systems: Lessons from Shopify Sidekick (2025) - Shopify