Enterprise-Grade RAG Systems | Harvey

This post details the technical considerations and implementation choices for building enterprise-grade Retrieval-Augmented Generation (RAG) systems at Harvey. It focuses on the critical role of vector databases, evaluating options like Postgres with PGVector and LanceDB Enterprise. Key technical challenges addressed include sparse vs. dense representations, performance and accuracy at scale, complex query structures, domain-specific data intricacies, and evaluation methodologies. The post highlights the requirements for enterprise-ready vector databases, emphasizing data privacy, security, and on-premise hosting. It also discusses the refinement of retrieval pipelines through collaboration with domain experts, citing the Tax AI Assistant as an example.