From Raw Documents to AI-Ready Data — Leo Platzer & Jeff Koss

This post contributes to agent harness engineering by demonstrating the generation of context files for coding agents using an SDK. It also significantly advances the understanding of data quality's impact on RAG performance by quantifying the negative effects of duplicates, conflicts, and staleness in large document corpora (80,000 SharePoint files). The talk quantifies that stale data can fill up to 80% of an agent's retrieved context and that cleaning this data roughly doubled recall in multi-hop RAG evaluations. It showcases AI-assisted taxonomies, metadata tagging with evidence and confidence scores, sensitive data detection, and quality checks for duplicates, conflicts, and freshness, all crucial for preparing unstructured data for AI applications.