Why we Built our own Cloud Agent Infrastructure | Harvey

This post details the rationale and technical considerations behind Harvey's decision to build and maintain its own cloud agent infrastructure. Key contributions include: 1. **Multi-Model Flexibility:** Architectural design to abstract away differences in agent harnesses, tool-call formats, and execution sandboxes across various AI models, enabling dynamic routing for conflict resolution, quality optimization, and cost control. 2. **Zero Data Retention (ZDR):** Architectural commitment to ZDR by ensuring customer data is not written to durable application storage by default, with agent sandboxes designed for lifecycle-bound transient working disks that are automatically cleaned up. This addresses the critical requirement for privileged and confidential legal data. 3. **Cost Optimization:** Implementation of fine-grained control over model routing and execution sandboxes to enable routing tasks to the most efficient model (including self-hosted open-source models) that meets quality thresholds, leading to significant cost reductions. The post also highlights the importance of owning the runtime for legal-specific layers, multi-cloud resilience, data residency, and sovereign deployments.