Reflections on ChatGPT and Claude’s Memory Systems: What It Takes to Enable On-Demand Conversational Retrieval

This post details how Milvus 2.6's hybrid dense-sparse retrieval, efficient scalar filtering, and tiered storage enable on-demand conversational retrieval for AI agents. It contrasts ChatGPT's precomputed context injection with Claude's RAG-style, on-demand retrieval, highlighting the architectural constraints (low latency tolerance, hybrid search, storage-compute separation) that make the latter practical with systems like Milvus.