TechPrufer
Get started
Harvey
Harvey logoHarvey · Feature trail

Document Processing Scalability

19 posts · Feb 2025 – Sep 2026

What this trail covers

Harvey's document processing capabilities have been significantly scaled to handle tens of millions of documents weekly, a 26x increase from the previous year. This was achieved by re-architecting the pipeline into distinct, independently scalable systems for extraction, chunking/embedding, and indexing. Key improvements include the adoption of a new Job Framework for durable and observable background work, the introduction of Unified Document Format (UDF) to manage intermediate formats efficiently, and enhanced AI models for classification and risk extraction. The platform now supports AI-powered due diligence workflows, including document classification, contract analysis, cross-document pattern recognition, and synthesis/reporting, enabling faster and more consistent analysis of large document sets in M&A and other transactions.

All Harvey trails
  1. Agentic Vault Search and Organization in HarveyIntroduces 'Agentic Vault Search' and 'Agentic Vault Organization' to enhance document discovery and structuring within Harvey's Vault system. Agentic Vault Search allows AI agents to locate relevant documents across entire Vaults without explicit folder or naming instructions, leveraging signals like folder location, naming conventions, and uploader identity. Agentic Vault Organization enables users to instruct agents in natural language to rename and reorganize files (e.g., by counterparty and agreement type), with a proposed plan and human confirmation step before execution. These features operate within existing permission and audit trail frameworks, and do not edit document content.
  2. Analyze the Full Scope of Legal Evidence in HarveyThis post details the integration of audio and video transcription, and enhanced image understanding into Harvey's document processing pipeline. This expands the system's ability to analyze multimodal evidence beyond text, supporting formats like M4A, MP3, WAV, WebM, FLAC, OGG for audio (up to 4GB/2 hours) and MP4, MOV, AVI, DAV, WebM for video (up to 2 hours). Image analysis now includes charts, graphs, and diagrams within documents. The system retains only the transcript, not the original audio, for privacy. This enables querying across entire sets of deposition videos and surfacing contradictions, tracing timelines, or pulling specific issues into Review Tables.
  3. Scaling Document Processing Across HarveyThis post details the infrastructure changes made to scale Harvey's document processing capabilities from handling under a million documents weekly to tens of millions. The core contribution is the architectural split of the document processing pipeline into three distinct, independently scalable systems: extraction, chunking/embedding, and indexing. This was enabled by migrating to a new Job Framework for durable and observable background work, introducing the Unified Document Format (UDF) to manage intermediate document representations, and optimizing the indexing hot path with a live vector database migration and the adoption of Arrow IPC for serialization/deserialization. The system now implements backpressure mechanisms to handle downstream rate limits and failures gracefully.
  4. How AI Speeds Up Legal Document Management for Legal TeamsThis post details how AI-powered document management transforms legal files into a searchable knowledge base for faster review, drafting, and analysis. It explains how modern systems organize, retrieve, compare, analyze, and apply documents across matters, moving beyond passive filing cabinets. The post highlights AI-assisted document review and due diligence, contract drafting and markup by leveraging negotiation history and precedent language, and litigation document management for organizing evidence and case knowledge. It emphasizes Harvey's integration with existing tools like Microsoft Word, Outlook, and DMS platforms to support these workflows. The post also discusses the importance of context-aware AI, matter-level analysis versus task-level automation, and the integration of firm-specific knowledge and governance models. It positions Harvey as a solution that turns document-heavy tasks into repeatable processes reflecting organizational playbooks and precedents.
  5. AI for eDiscovery: Faster Document Review Without the RiskThis post details the integration of AI (ML, NLP, GenAI) into Harvey's eDiscovery document review process. It explains how AI addresses the scale and complexity of modern data by improving consistency, context, and prioritization. Specific AI applications include categorizing and prioritizing documents, summarizing content, flagging privileged information, and accelerating early case assessment. The post also highlights the use of Technology-Assisted Review (TAR) with Continuous Active Learning (CAL), Natural Language Processing (NLP) for conceptual search and entity extraction, and Generative AI (GenAI) for document summaries, privilege log drafting, deposition preparation, and narrative timelines. It emphasizes risk mitigation strategies such as grounding AI outputs in document sets, ensuring proportionality and validation, and maintaining privilege and confidentiality through specific platform controls. The post also outlines key features to look for in an eDiscovery AI platform, including data governance, validation and defensibility, customization, workflow integration, and purpose-built legal AI.
  7. Building Image Understanding for Legal Documents | HarveyIntroduced an on-demand vision system for legal documents that enables AI agents to reason over charts, diagrams, and complex layouts at query time. This system avoids expensive, lossy, and potentially inaccurate indexing-time analysis of all images. Key components include agent-driven invocation, smart page finding via text search, high-fidelity rendering service, and structured visual reasoning with confidence levels. The approach prioritizes text-based analysis first, invoking vision only when necessary. Engineering challenges addressed include scale (narrowing candidate pages to 2-3 in milliseconds), consistency (converting documents to a consistent format and rendering at high DPI), cost (cutting waste by processing only necessary images), and graceful degradation (reporting what can and cannot be determined). Key learnings emphasize that foundation models are effective with text alone, and tool description tuning is critical for agent recall and response quality.
  8. The Guide to AI-Powered Due Diligence for M&A ProfessionalsThis post details the application of AI, specifically NLP, ML, and generative AI, to automate and enhance due diligence processes in M&A transactions. It describes how AI models can classify documents, extract key provisions, identify cross-document patterns, and generate structured reports. The post highlights the benefits of consistent analysis, reduced human error, and accelerated timelines compared to traditional manual review. It also touches upon the integration of these AI capabilities into various stages of the M&A deal lifecycle, from pre-deal research to data room organization and final reporting.
  9. Building an Agent for Complex Document Drafting and EditingThis post details the architectural shift from a one-shot sub-agent system to an in-memory agent that iteratively builds and edits Word documents. Key technical contributions include: a proprietary OOXML library for full feature support and active maintenance, an agent that prioritizes legal text while exposing minimal OOXML for markup preservation, an in-memory document representation for iterative editing with optimized tools, a get_diff tool for agent self-review, and a reconciliation step for merging edits from parallel sub-agents. The system now dynamically gathers context and supports advanced multi-step operations on lists, tables, and styles, leading to significant improvements in edit quality and user engagement.
  10. Stay in Flow From Document Intake to Final DeliverableThis post details the integration of in-platform Word editing capabilities into Harvey's document processing workflow. This allows users to review tracked changes, redlines, and incorporate additional context from vaults and knowledge sources directly within the platform, without leaving Harvey. The system preserves original formatting and enables downloading client-ready deliverables in docx format. This extends the continuous flow from document intake and analysis to final drafting and polishing, reducing manual reformatting and context switching.
  11. Building a New File Ingestion System to Scale Firm KnowledgeThis post details the development of a new file ingestion system to scale firm knowledge by enabling folder uploads and one-way sync from document management systems. It introduces an asynchronous workflow architecture using Temporal for orchestration, focusing on resilience through granular activity design for crawling and isolated file downloads. A custom in-house rate limiter built on Redis was implemented to manage external API quotas. A diffing algorithm using external IDs, hash-based comparison, and timestamp fallbacks ensures correctness for sync operations. Performance optimizations include aggressive batching, connection pooling, parallelization, using blob storage for intermediate states, careful task queue design, and immediate cleanup of temporary data.
  12. Using Agents to Scale Harvey’s Knowledge SourcesThis post details the creation of 'The Data Factory,' an autonomous pipeline of AI agents designed to scale Harvey's knowledge sources. Key technical contributions include: an 'Intake engine' with a Sourcing Agent for automated legal infrastructure mapping and a Legal Review Agent for accelerated compliance review (doubling attorney throughput); a declarative configuration layer replacing hand-tuned systems, allowing jurisdictions to be defined by domain lists, filter hierarchies, permissions, and agent instructions, enabling a single reasoning agent to access parameterized tools across jurisdictions; and a four-step evaluation pipeline (Answer-first scenario generation, Production simulation, Trace validation, Multi-agent quality assessment) using specialized agents (URL Classification, Citation Quality, Answer Quality, Presentation, Decision) to rigorously test agent performance on new sources without human-labeled datasets.
  13. Enabling Document-Wide Edits in Harvey’s Word Add-InIntroduced document-wide editing capabilities for the Word Add-In, allowing users to edit 100+ page documents with a single query. This was achieved by developing an orchestrator-subagent architecture to decompose complex edits into targeted tasks, mitigating the 'lost in the middle' effect and improving thoroughness. The implementation involved a reversible mapping between OOXML objects and natural language to handle document complexity and preserve formatting. Scalable offline evaluation methods were developed to test numerous model combinations and validate the effectiveness of representing tracked changes with <ins> and <del> tags.
  14. Low Latency Redline Detection via Vision ModelsIntroduced a low-latency redline detection system using custom vision models to identify redlined documents. This system converts PDFs to images using pyvips, trains a small, purpose-built CNN (preserving performance while meeting sub-second latency on common CPUs), and optimizes for speed and memory through sampling techniques (random sampling, RGB and edge detection filters). This enables selective application of resource-intensive span extraction processing only to documents identified as redlined, achieving zero additional latency to the user experience.
  15. Enterprise-Grade RAG Systems | HarveyThis post details the technical considerations and implementation choices for building enterprise-grade Retrieval-Augmented Generation (RAG) systems at Harvey. It focuses on the critical role of vector databases, evaluating options like Postgres with PGVector and LanceDB Enterprise. Key technical challenges addressed include sparse vs. dense representations, performance and accuracy at scale, complex query structures, domain-specific data intricacies, and evaluation methodologies. The post highlights the requirements for enterprise-ready vector databases, emphasizing data privacy, security, and on-premise hosting. It also discusses the refinement of retrieval pipelines through collaboration with domain experts, citing the Tax AI Assistant as an example.