Change Data Capture vs Batch ETL for AI Agents

This post advocates for Change Data Capture (CDC) over batch ETL for keeping AI agent context fresh. It details log-based CDC (PostgreSQL, MySQL binlog via Debezium) as superior to trigger-based and polling methods due to lower latency, capture of deletes, and minimal impact on write transactions. It highlights the structural staleness of batch ETL, citing examples from Uber and Netflix where stream processing significantly reduced data freshness lag. The post emphasizes that stale context leads to agent errors, liability (Air Canada example), and degraded output quality (context poisoning, version-aware RAG benchmarks). It positions CDC as a critical engineering choice for reliable AI agent operation.