TechPrufer
Get started
Milvus
Milvus logoMilvus · Feature trail

Multi-source File-like Search (MFS)

2 posts · Dec 2025 – Jul 2026

What this trail covers

MFS provides a unified, file-like interface for agents to search and browse across heterogeneous data sources (code, docs, tickets, databases, SaaS tools, object stores). It combines semantic search with progressive disclosure (tree, ls, cat) to enable agents to efficiently retrieve and verify context, reducing token costs and improving accuracy. Provence and X Provence are introduced as lightweight context pruning models that can be integrated into RAG pipelines to filter irrelevant information before it reaches the LLM, improving response quality and reducing latency. Open Tag is an example application built on MFS, enabling an agent to review PRs and other work artifacts in Slack. This post details tools that enable context pruning and its evaluation.

All Milvus trails
  1. Open Tag: An Open-Source Claude Tag for Claude Code and CodexThis post introduces Open Tag, an open-source implementation of the Claude Tag pattern built on top of the MFS (Multi-source File-like Search) framework. It details how Open Tag leverages MFS to provide agents with a reliable way to find, verify, and use context across multiple systems without dumping everything into the prompt. The post explains the MFS architecture, which exposes heterogeneous sources through a stable, file-like URI interface and combines search with progressive disclosure (tree, ls, cat) for efficient context retrieval and verification. It also covers the agent-driven installation and configuration process for Open Tag and demonstrates its use in reviewing a PR.
  2. 7 Best Open-Source Tools for Claude Code Context ManagementThis post reviews seven open-source tools (RTK, Context Mode, code-review-graph, Token Savior, Caveman, claude-context, memsearch) that address context defocus in LLM workflows. It explains how these tools manage terminal logs, tool outputs, codebase exploration, file reads, model verbosity, and session memory. The post explicitly maps these context management strategies to vector database design, vector similarity search, and retrieval systems, highlighting how techniques like semantic code retrieval and progressive disclosure (similar to MFS) improve context quality for AI agents.
  3. Harness Engineering: The Execution Layer AI Agents Actually NeedThis post details the retrieval problem within AI agent harnesses, specifically the need for hybrid search that combines semantic (embedding-based) and keyword (exact-match) retrieval. It explains how agents require both types of search simultaneously to find relevant information from evolving knowledge bases, such as identifying design principles semantically and finding specific function references via keywords. The post emphasizes that this retrieval capability is a silent dependency for effective agent harnesses, enabling agents to navigate and utilize information from diverse sources.
  4. We Read Claude Code's Leaked Source. Here's How Its Memory Actually WorksThis post analyzes the memory architecture of Claude Code, identifying limitations such as a 200-line index cap, keyword-only retrieval, lack of reasoning capture, and memory being locked within the agent. It then introduces memsearch, a persistent memory layer for AI coding agents. memsearch captures conversations, stores them as dated Markdown files, and uses Milvus for a hybrid semantic and BM25 keyword search with RRF ranking. It supports L1 semantic search with previews, L2 full context retrieval, and L3 reasoning extraction, enabling cross-agent memory portability and richer recall.
  5. LLM Context Pruning: A Developer’s Guide to Better RAG and Agentic AI ResultsThis post introduces Provence and X Provence as lightweight context pruning models for RAG pipelines. It details four common failure modes in long-context systems (Context Clash, Context Confusion, Context Distraction, Context Poisoning) and explains how context pruning addresses these by filtering irrelevant information. Provence is described as a cross-encoder model trained to perform document-level relevance scoring and token-level relevance labeling simultaneously. It can be integrated into existing RAG pipelines with minimal overhead, and its pruning behavior can be tuned. The post also outlines quantitative experiments comparing Provence, X Provence, and OpenSearch Semantic Highlighter on the WikiText-2 dataset.
  7. Reflections on ChatGPT and Claude’s Memory Systems: What It Takes to Enable On-Demand Conversational RetrievalThis post details the architectural differences in memory systems between ChatGPT (precomputed context injection) and Claude (RAG-style, on-demand retrieval). It explains how Claude's approach necessitates a vector database capable of low-latency, hybrid search and storage-compute separation. It explicitly calls out Milvus 2.6's features (hybrid dense-sparse retrieval, efficient scalar filtering, tiered storage) as enabling this on-demand conversational retrieval at scale, addressing the constraints of latency, hybrid search, and storage-compute separation.
  8. Keeping AI Agents Grounded: Context Engineering Strategies that Prevent Context Rot Using MilvusThis post details context engineering strategies to combat context rot in long-running AI agent conversations. It explains the O(n^2) complexity of Transformer attention and the out-of-distribution nature of long contexts for LLMs. It proposes three approaches: Just-in-Time (JIT) retrieval, pre-retrieval using vector search (Milvus), and a hybrid approach. The hybrid approach combines Milvus-powered vector search for stable knowledge with agent-driven JIT exploration for dynamic information. Milvus is highlighted for its support of vector search + scalar filtering and incremental updates, enabling efficient retrieval for both stable and dynamic data sources in agent workflows.