Dynamic chunking for RAG: building context infrastructure that adapts

This post introduces dynamic chunking strategies for RAG to improve context infrastructure. It details the breakdown of fixed-size chunking due to document and query variations, and explores content-adaptive strategies: semantic chunking (splitting by meaning shifts), late chunking (embedding full documents then segmenting), hierarchical chunking (parent-child indexing), proposition-based chunking (atomic factoids), and agentic/query-aware chunking (LLM-driven boundary selection or multi-granularity routing). It also discusses real-time retrieval improvements via re-ranking, adaptive routing, query rewriting (including model racing), and optimal chunk ordering to combat context confusion and rot. Redis Iris is positioned as the underlying vector storage and retrieval layer for these dynamic RAG operations.