TechPrufer
Get started
Redis
Redis logoRedis · Feature trail

Vector Search Indexing and Operations

8 posts · Jun 2026 – Sep 2026

What this trail covers

Redis Search indexes can now live on Flex tiered storage in Redis Cloud Pro, allowing for large-scale search on terabyte-scale datasets with a significantly reduced RAM footprint. This enables use cases like comprehensive fraud screening, full corpus RAG pipelines, and unified caching/search layers. The same Search API and client libraries are used, and users can dynamically adjust the RAM-to-Flash ratio without re-indexing. Query Performance Factor (QPF) is used to maintain acceptable latency. This post extends this by detailing how Redis Flex (RAM + SSD tiering) can be applied to agentic workloads and feature stores to manage increasing memory costs driven by AI, allowing terabyte-scale datasets to be managed with a lower RAM footprint by dynamically adjusting the RAM-to-SSD ratio.

All Redis trails
  1. Scale your workloads without paying all-RAM prices | RedisThis post introduces Redis Flex, a tiering solution combining RAM and SSD, as a strategy to mitigate rising memory costs impacting in-memory databases, particularly for AI workloads like agents and feature stores. It explains how tiering works by keeping hot data in RAM and less frequently accessed data on SSD, allowing for terabyte-scale datasets with a reduced RAM footprint and significant cost savings (up to 80% less per gigabyte). The post highlights that Flex supports dynamic adjustment of the RAM ratio without code changes and mentions its extension to Redis Search for storing large indexes on SSD (in preview).
  2. Your data is growing. Your RAM bill doesn't have to. | RedisThis post introduces Redis Flex, a tiered storage solution that combines RAM and SSD to manage growing datasets cost-effectively. It addresses the rising cost of memory driven by AI workloads by allowing users to store less frequently accessed data on SSDs while keeping hot data in RAM. The key technical innovation is the ability to move both keys and values to SSD, optimizing RAM usage for active data. This enables terabyte-scale datasets with configurable RAM ratios (10-50%) and significant cost savings (up to 80% less per gigabyte). Flex also now supports Redis Search indexes on SSD (in preview), further extending its cost-saving capabilities to search workloads.
  3. Search on Flex comes to Redis Cloud Pro: same Search API, a fraction of the RAM | RedisIntroduces 'Search on Flex' for Redis Cloud Pro, enabling Redis Search indexes to utilize Flex tiered storage (RAM and SSD). This reduces the memory footprint of search indexes to roughly 10% of in-memory equivalents, making terabyte-scale search practical. It leverages Query Performance Factor (QPF) to mitigate latency increases from disk reads, targeting a 10x latency increase at 10% RAM cost. The feature maintains the existing Redis Search API and client libraries. It supports HASH, TEXT, TAG, and VECTOR index types, with NUMERIC, GEO, JSON, FT.AGGREGATE, FT.HYBRID, and background indexing planned for future parity.
  4. Metadata Filtering: Boost Vector Search Precision at ScaleThis post details the critical role of metadata filtering in enhancing vector search precision, especially as datasets grow. It explains the difference between pre-filtering and post-filtering, and where each breaks down at scale. The post introduces hybrid queries, specifically mentioning Reciprocal Rank Fusion (RRF) and Redis's FT.HYBRID command, as a way to combine keyword search and vector similarity. It emphasizes the benefits of keeping filters, text indexes, and vectors within a single engine to avoid data synchronization issues and maintain consistency. Best practices for metadata filtering, such as limiting filterable indexes and treating tenant/permission filters as security controls, are also provided.
  5. Vector Search: A Practical Guide for DevelopersThis post provides a practical guide to production vector search, detailing index trade-offs (FLAT vs. HNSW) and their impact on recall, speed, and memory. It explains distance metrics (cosine, Euclidean, dot product) and their relevance to embedding models. The post outlines common production patterns like RAG, hybrid search, semantic caching, recommendations, and agent memory, highlighting how vector search fits into these architectures. It also addresses key failure modes such as memory constraints with large datasets (leading to quantization) and recall degradation due to metadata filtering, along with the challenge of embedding drift when models are updated.
  7. Vector Search Database: News & 2026 GuideThis post updates the understanding of vector search databases by highlighting their integration into existing data management platforms, the trend of vectors as a first-class data type, the rise of hybrid search, and the importance of metadata filtering. It discusses the impact of RAG, semantic caching, and agent memory on driving demand. The post also provides updated evaluation criteria for 2026, focusing on hybrid search, filtered query performance, tail latency, and operational footprint, and positions Redis as a solution that maps to these criteria by offering vector search alongside its other capabilities.
  8. Vector embeddings & language: how models turn words into geometryThis post explains the linguistic challenges and evolution of creating vector embeddings from text. It details how models learn to represent meaning as geometry, moving from static word embeddings (word2vec, GloVe) to contextual embeddings (BERT) that account for ambiguity and context. It describes pooling strategies for sentence/document embeddings and the encoding of relationships (e.g., 'capital of') and biases within vector spaces. The post also introduces Approximate Nearest Neighbor (ANN) algorithms like HNSW for efficient retrieval and highlights Redis's integration of vector search with operational data and AI context engines like Redis Iris.
  9. Reciprocal Rank Fusion: How Hybrid Search Really WorksThis post introduces Reciprocal Rank Fusion (RRF) as a method for combining ranked lists from different search systems, specifically keyword and vector search. It explains the limitations of directly adding raw scores from BM25 and vector search due to incompatible scales and drift, and how RRF, by using document ranks, sidesteps these issues. RRF's formula is presented, along with an example demonstrating how consensus across retrievers outweighs a single top rank. The post highlights RRF's unsupervised nature and its application in hybrid search, recommendation systems, and other ranking scenarios. It also stresses the importance of retrieval speed over the fusion step in production pipelines, particularly for latency-sensitive applications and AI agents. The post notes that Redis Search implements RRF for hybrid search within a single query, serving as a fast retrieval layer for systems like Redis Iris.
  10. Top vector database alternatives for RAG pipelinesThis post analyzes the landscape of vector database alternatives for RAG pipelines, comparing purpose-built vector databases (Pinecone, Weaviate, Milvus, Qdrant, Chroma) and pgvector against Redis's unified real-time data platform. It highlights the architectural trade-offs of standalone vector databases versus integrated solutions, emphasizing the benefits of consolidation for infrastructure overhead, cost, and scope. Specific advantages of Redis are detailed, including its sub-millisecond latency, hybrid search capabilities, semantic caching (LangCache) for LLM cost reduction, and deployment flexibility. The post also provides a high-level overview of the strengths and weaknesses of each alternative, with a focus on when Redis is the most suitable choice.
  11. Top Reranking Models to Boost RAG Accuracy in 2026This post introduces reranking models as a critical layer in RAG pipelines to improve accuracy by reordering retrieved documents. It details the function of rerankers in addressing first-stage retrieval's limitations and mitigating 'context rot' by prioritizing relevant chunks. The post categorizes rerankers into cross-encoder, LLM-based, and late-interaction models, discussing their respective trade-offs in terms of speed, cost, and complexity. It provides guidance on selecting rerankers based on accuracy vs. latency budgets, context length, and multilingual requirements. The post also stresses the importance of a strong first-stage retrieval (e.g., hybrid retrieval) to feed the reranker with quality candidates and mentions Redis's role in providing the necessary data infrastructure for these operations.
  12. RAG Debugging Guide: Fix Retrieval Errors FastThis post provides a practical guide to debugging RAG retrieval errors, categorizing common failures into five stages: retrieval, ranking, grounding, freshness, and latency. It details specific causes and solutions for each stage, including: 1. Retrieval errors due to poor chunking, vocabulary mismatch, or reliance on exact identifiers, recommending better chunking strategies and hybrid search. 2. Ranking errors where correct chunks are buried, addressed by cross-encoder reranking and metadata filtering. 3. Hallucinations despite good retrieval, linked to grounding issues and context engineering, emphasizing evidence placement and avoiding context rot. 4. Stale or duplicate results due to index freshness issues, highlighting the benefits of incremental updates and real-time data ingestion. 5. Slow retrieval under load, which can be misdiagnosed as accuracy problems, emphasizing the need for robust infrastructure and error handling.
  13. Vector Embeddings Explained: Theory to Real-World UseThis post provides a foundational explanation of vector embeddings, detailing their theoretical underpinnings, how they are created by neural networks (e.g., Word2Vec, BERT, CLIP), and the process of similarity search using distance metrics (cosine similarity, dot product, Euclidean distance) and Approximate Nearest Neighbor (ANN) algorithms like HNSW. It outlines key production use cases including semantic search, recommendations, RAG, semantic caching, and agent memory, highlighting the importance of model choice and the need for re-generating embeddings when models change. The post also briefly touches on the infrastructure challenges of storing and querying large vector datasets.
  14. Dynamic chunking for RAG: building context infrastructure that adaptsThis post introduces dynamic chunking as a strategy to improve RAG retrieval quality by adapting chunk boundaries and selection based on content and queries, moving beyond fixed-size chunking. It details content-adaptive strategies like semantic, late, hierarchical, proposition-based, and agentic chunking, along with their tradeoffs. It also covers query-time adaptations such as re-ranking, adaptive routing, and query rewriting to improve retrieval in real-time, especially for follow-up questions. The post emphasizes the importance of the underlying vector indexing and retrieval infrastructure, like Redis Iris, in supporting these dynamic strategies.
  15. Comparing the best open source vector databases (2026)This post compares various open-source vector databases, highlighting Redis's approach as a unified platform for vector search, caching, and operational data. It details Redis's vector search architecture (HNSW, FLAT), billion-vector benchmark performance (90% precision at ~200ms median latency), hybrid search capabilities (FT.HYBRID command with Reciprocal Rank Fusion and linear combination), and semantic caching with Redis LangCache (up to 70% savings). It contrasts Redis's deployment options (Cloud, Software, Open Source) and unified approach with specialized databases like Milvus, Weaviate, Qdrant, Chroma, pgvector, and Faiss, emphasizing the trade-offs in operational complexity, feature sets, and integration.
  16. Knowledge graph RAG: structured retrieval for AI agentsIntroduces knowledge graph RAG (GraphRAG) as an advancement over traditional vector RAG. Explains how GraphRAG models data as entities and relationships, enabling multi-hop retrieval for AI agents. Details the four-step GraphRAG pattern: data extraction, query entity linking, graph traversal, and response generation. Discusses the limitations of vector search for relationship-path queries, whole-corpus questions, and dense domains. Highlights the accuracy and hallucination reduction benefits of multi-hop retrieval. Addresses the 'freshness problem' in knowledge graphs caused by batch indexing and proposes near-real-time updates. Positions structured retrieval within an agent's context path, emphasizing tiered memory management (short-term, long-term, graph-based).
  17. Context Graphs vs. Vector Search: When RAG Falls ShortThis post introduces context graphs as a complementary retrieval mechanism to vector search for RAG pipelines. It explains the limitations of vector-only RAG for multi-hop questions, fragmented context, and exact match retrieval. It details the components of context graphs (entity/relationship extraction, knowledge graph construction, graph-aware retrieval) and their advantages in specific query types. The post advocates for a dual-channel retrieval architecture that combines vector and graph search, highlighting Redis's capabilities in vector search (Query Engine, FT.HYBRID) and its role in supporting the vector half of this pattern. Redis Iris is presented as a layer that can integrate with this dual-channel design.