BlogsQdrantData Ingestion and Integration

Data Ingestion and Integration

Data Ingestion and Integration

44
posts
2024–2026

Qdrant Hybrid Cloud is launched, enabling deployment in any environment (cloud, on-premise, edge). This offering integrates with various infrastructure providers (OCI, Red Hat OpenShift, Vultr, DigitalOcean, OVHcloud, Scaleway, Civo, STACKIT) and AI frameworks (LlamaIndex, LangChain, Haystack, Airbyte, JinaAI, Aleph Alpha). Tutorials demonstrate building AI applications like RAG systems, chatbots, and recommendation engines on partner platforms, emphasizing flexibility, data sovereignty, and con. This post specifically details the integration with Jina AI's embedding models for enhanced RAG capabilities, emphasizing seamless deployment, scalable vector search, and cost efficiency.

2026

How Sapu Indexed 28 Million PubMed Abstracts to Accelerate Cancer Research with Qdrant - Qdrant

5/12/2026

This post details Sapu's use of Qdrant to index 28 million PubMed abstracts into a single collection, enabling hybrid retrieval and per-document parallel querying. The architecture involves OpenAI's text-embedding-3-large model and LangChain for ingestion. Qdrant Cloud Premium was adopted for reliability and compliance. Future plans include evaluating Qdrant Edge for on-device vector search in air-gapped environments and Qdrant Cloud Inference for embedding pipelines.

How Data Graphs Built a True Hybrid Graph RAG Platform - Qdrant

4/22/2026

This post details Data Graphs' implementation of a 'True Hybrid Graph RAG Platform' using Qdrant. Key technical contributions include: 1. Real-time embedding from a graph database into Qdrant collections with rich metadata for filtering. 2. An agentic AI layer that leverages Qdrant's payload filtering and graph queries to dynamically select the best retrieval path based on schema analysis and prompt requirements. 3. Blended retrieval from Qdrant (for semantic similarity) and the graph database (for empirical, structured data) with verifiable provenance. Qdrant's Hybrid Cloud deployment and payload filtering capabilities were critical for integrating with existing infrastructure and data models.

How GlassDollar improved high-recall sourcing by migrating from Elasticsearch to Qdrant - Qdrant

3/4/2026

GlassDollar migrated from Elasticsearch to Qdrant for their high-recall sourcing product. This migration improved retrieval speed, reduced infrastructure costs by 40%, and enabled more effective query expansion. The system leverages the Qdrant Node.js SDK for integration within their existing Node.js/TypeScript backend services. The improved retrieval quality led to a 3x increase in user engagement (bookmarks).

Convolve 4.0 - IIT Hackathon Winners - Qdrant

2/27/2026

This post highlights the use of Qdrant in three winning projects from the Convolve 4.0 IIT Hackathon: Masthishq (multimodal AI agent for patient assistance), SignalWeave (temporal AI memory system for trend detection), and Demeter (autonomous multi-agent system for hydroponic farm management). All projects leverage Qdrant as a persistent vector memory layer for storing embeddings, enabling retrieval, and facilitating long-term memory and reasoning in AI systems.

How My AskAI Built Self-Improving Support Agents - Qdrant

2/25/2026

This post details how My AskAI built its AI customer support agent platform using Qdrant Cloud. It highlights the transition to a customer support focus, the adoption of embeddings and RAG, and the reasons for choosing Qdrant (scalability, integrations, developer experience). The post also discusses the benefits of Qdrant Cloud for scaling and managing hybrid search, and introduces the concept of self-learning support agents where human agent responses are indexed in Qdrant to continuously update the knowledge base.

How Kakao Built an AI-Powered Internal Service Desk with Qdrant - Qdrant

1/27/2026

This post details Kakao's implementation of an AI-powered internal service desk using Qdrant as a RAG system. It highlights the use of Qdrant's hybrid search (dense and sparse vectors with RRF) to handle both semantic and exact-match queries across technical documentation and historical Q&A data. The implementation uses Named Vectors, scalar quantization (INT8) for memory optimization, and an asynchronous Python client for high concurrency. Qdrant is self-hosted on Kubernetes with a replication-based setup for high availability. The system supports approximately 1 million vectors and has reduced end-to-end response times to under 30 seconds.

2025

How Xaver scaled personalized financial advice with Qdrant - Qdrant

11/13/2025

This post details how Xaver built an AI knowledge engine using Qdrant for personalized financial advice. It describes a two-tier retrieval architecture: a Condensed Knowledge Base (CKB) for common questions and a Full Knowledge Base for detailed queries. Qdrant was chosen for its high performance at low latency, developer simplicity, and flexible deployment, enabling sub-second retrieval for conversational use cases and improving advisor efficiency.

Qdrant for Research: The Story Behind ETH & Stanford’s MIRIAD Dataset - Qdrant

7/23/2025

This post details the use of Qdrant in the creation of the MIRIAD dataset, a large-scale medical QA dataset. It highlights Qdrant's role in powering the dataset's storage and RAG experiments, emphasizing its simplicity of use, speed, scalability, and open-source nature as key factors for researchers. The post also mentions the development of MIRIAD Atlas, an interactive map visualizing UMAP-based dimensionality-reduced embeddings, similar to Qdrant's WebUI visualization capabilities. Future plans include migrating MIRIAD Atlas to Qdrant for full-scale dataset serving.

How &AI scaled global legal retrieval with Qdrant - Qdrant

7/15/2025

&AI scaled global legal retrieval by using Qdrant Cloud to manage billions of patent documents. They employed OpenAI's embedding models to transform data into vectors and indexed them in Qdrant. Key technical decisions included using scalar quantization for a balance of retrieval speed and accuracy, and leveraging Qdrant's multi-tenancy feature for data isolation between firms. The architecture focuses on retrieval-first, using Qdrant's payload filtering and HNSW for refined search results.

Introducing Qdrant Cloud Inference - Qdrant

7/15/2025

Introduces Qdrant Cloud Inference, a new capability that integrates model inference directly into Qdrant Cloud. This allows users to generate, store, and index embeddings within a single API call, eliminating the need for separate inference infrastructure and reducing latency. It supports multiple embedding models for multimodal and hybrid search, including CLIP-style models for text and images, and offers free token allowances for paid users. Inference is automatically enabled for paid accounts with version 1.14.0 or higher.

How FAZ unlocked 75 years of journalism with Qdrant - Qdrant

7/3/2025

FAZ has built a hybrid search engine using Qdrant and Azure OpenAI for their 75-year journalism archive, enabling semantic search with rich metadata filtering and real-time updates. They are planning to integrate sparse vectors for hybrid search capabilities.

GraphRAG: How Lettria Unlocked 20% Accuracy Gains with Qdrant and Neo4j - Qdrant

6/17/2025

Lettria's GraphRAG system integrates Qdrant with Neo4j for enhanced accuracy in complex document intelligence, achieving >20% accuracy gains by combining vector search with graph-based semantic understanding. Key technical challenges addressed include maintaining data consistency between Qdrant and Neo4j through a custom ingest transaction mechanism with snapshotting and conflict resolution, and optimizing query performance via payload flattening and disk-based caching for large-scale vector datasets (>100M vectors).

​​Introducing the Official Qdrant Node for n8n - Qdrant

6/9/2025

Introduces an official Qdrant node for n8n, allowing direct integration of Qdrant's features (batch upserts, updates, hybrid search, recommendations, discovery search, distance matrix API) into n8n workflows without relying on HTTP nodes. Demonstrates a hybrid search example using Reciprocal Rank Fusion (RRF) and highlights support for advanced features like the Score-Boosting Reranker.

How Qovery Accelerated Developer Autonomy with Qdrant - Qdrant

5/27/2025

This post details how Qovery integrated Qdrant Cloud into their DevOps automation platform to power an AI-driven DevOps Copilot. They utilized Qdrant for its real-time indexing and low-latency queries to manage a corpus of vectors derived from historical usage data, enabling developers to manage infrastructure tasks via natural language. The integration focused on seamlessness, scalability, and performance, allowing Qovery to reduce reliance on specialized DevOps expertise and accelerate infrastructure task execution.

How Dust Scaled to 5,000+ Data Sources with Qdrant - Qdrant

4/29/2025

This post details Dust's successful scaling to over 5,000 data sources by migrating to Qdrant. Key technical contributions include: consolidating thousands of individual collections into a few shared, multi-tenant ones using Qdrant's sharding and payload filtering; implementing scalar quantization to reduce vector storage by 4x and enable in-memory data, drastically improving search latency; building a custom `DustQdrantClient` to abstract Qdrant operations; and deploying Qdrant in isolated VPCs across US and EU regions for high availability and compliance. The post also highlights the architectural flexibility gained by grouping data sources by embedder, enabling smoother migrations and model experimentation.

How SayOne Enhanced Government AI Services with Qdrant - Qdrant

4/28/2025

This post details SayOne's experience using Qdrant for government AI services, comparing it to Pinecone. Key technical aspects highlighted include Qdrant's low latency, simplified deployment, hybrid search capabilities, and advanced payload filtering, which were crucial for meeting government requirements for data privacy, scalability, and responsiveness. The post emphasizes Qdrant's ability to host deployments closer to user locations and its flexibility for cloud-based and on-premise environments.

How Mixpeek Uses Qdrant for Efficient Multimodal Feature Stores - Qdrant

4/8/2025

Mixpeek adopted Qdrant as their specialized feature store for multimodal data, migrating from MongoDB. This migration resulted in 40% faster query times for retrievers (from ~2.5s to 1.3-1.6s) due to Qdrant's prefetching and parallel retrieval capabilities. Hybrid retriever implementations were simplified, reducing code by 80% through Qdrant's native Reciprocal Rank Fusion (RRF) support and multi-vector indexing. SageMaker feature extraction workflows saw a 50% reduction in query overhead. Qdrant's payload filtering facilitated Mixpeek's taxonomy implementation, and batch vector search streamlined document clustering.

Vibe Coding RAG with our MCP server - Qdrant

3/21/2025

This post details the integration of Qdrant with the Model Context Protocol (MCP) server to enhance AI coding assistants. It explains how the `mcp-server-qdrant` acts as a semantic memory layer, enabling AI agents to store and retrieve code snippets and documentation using `qdrant-store` and `qdrant-find` tools. The post provides configuration examples for Claude Code, demonstrating how to set up tool descriptions for frontend code generation and storage, and lists key environment variables for the MCP server configuration, including embedding provider and model selection.

How Deutsche Telekom Built a Multi-Agent Enterprise Platform Leveraging Qdrant - Qdrant

3/7/2025

Deutsche Telekom built a multi-agent enterprise platform (LMOS) as a PaaS for AI agents, leveraging Qdrant for vector search and retrieval. They chose Qdrant for its simplicity, Rust efficiency, memory management, and operational fit for multitenancy, contrasting it with previous solutions that had higher operational overhead and downtime during scaling. The platform reduced agent development time from 15 days to 2 and now processes over 2 million conversations.

How to Build Intelligent Agentic RAG with CrewAI and Qdrant - Qdrant

1/24/2025

This post details the integration of Qdrant with CrewAI for building agentic RAG systems. It includes the development of a custom Qdrant storage implementation for CrewAI's entity and short-term memory, and outlines strategies for loading and semantically chunking data from sources like Obsidian notes into Qdrant for retrieval.

2024

Voiceflow & Qdrant: Powering No-Code AI Agent Creation with Scalable Vector Search - Qdrant

12/10/2024

This post details how Voiceflow integrated Qdrant into their no-code AI agent creation platform. Key technical contributions include Voiceflow's migration from Pinecone to Qdrant due to scaling capabilities and infrastructure support, their RAG pipeline setup involving data upload, ingestion strategies, and metadata tagging, and their specific usage of Qdrant features like Product Quantization and chunking search. They also highlight the architectural choice of node pools for multitenancy and the use of Private Link for security compliance.

Advanced Retrieval with ColPali & Qdrant Vector Database - Qdrant

11/5/2024

Introduces ColPali, a multimodal retrieval approach that uses Vision Language Models (VLMs) to process document images directly, creating multi-vector embeddings from both visual and textual content. Details the integration of ColPali with Qdrant, including the use of Binary Quantization for optimizing storage and computational load, and presents results showing a 2x faster search time compared to Scalar Quantization while maintaining accuracy.

Qdrant 1.12 - Distance Matrix, Facet Counting & On-Disk Indexing - Qdrant

10/8/2024

Introduced Distance Matrix API for pairwise vector distance calculations with 'pairs' and 'offsets' output formats. Added Facet API for counting unique values in payload fields, with an 'exact' parameter for precise counts. Enabled 'on_disk' option for text and geo indexes to reduce memory footprint. Integrated Distance Matrix API into the Graph UI for data visualization.

Qdrant and Shakudo: Secure & Performant Vector Search in VPC Environments - Qdrant

9/23/2024

This post details the partnership between Qdrant and Shakudo to offer Qdrant Hybrid Cloud within Shakudo's VPC environments. It highlights the technical aspects of deploying Qdrant on Kubernetes within a private cloud for enhanced data sovereignty, security, and performance. The integration leverages Qdrant's HNSW indexing for efficient handling of large vector datasets and its Kubernetes-native architecture for automated deployments and cluster management within Shakudo's pre-configured AI infrastructure.

Data-Driven RAG Evaluation: Testing Qdrant Apps with Relari AI - Qdrant

9/16/2024

This post details the integration of Qdrant with Relari AI for data-driven RAG evaluation. It introduces two key evaluation methods: Top-K parameter optimization and Auto Prompt Optimization. The post provides a practical example using the GitLab legal policies dataset, demonstrating how to build a RAG app with Qdrant, FastEmbed, and LangChain, log retrieval results for different Top-K values, and submit these results to Relari for performance analysis using metrics like Precision/Recall and Rank-Aware methods. It also outlines the setup for Auto Prompt Optimization by analyzing prompts and setting up a base prompt.

Qdrant 1.11 - The Vector Stronghold: Optimizing Data Structures for Scale and Efficiency - Qdrant

8/12/2024

Qdrant 1.11 introduces defragmentation of tenant storage to co-locate vectors from the same tenant for improved read performance and cost efficiency, configurable via `is_tenant=true` on payload indexes. It adds `on_disk=true` support for payload indexes to offload cold data, improving memory management for large datasets. A new UUID datatype for payload indexes reduces memory overhead by storing UUIDs as 16-byte values. The Query API now supports a Groups endpoint for result aggregation by payload fields, random sampling, and distribution-based score fusion. New Web UI tools include a Search Quality Tool and a Graph Exploration Tool.

Qdrant 1.10 - Universal Query, Built-in IDF & ColBERT Support - Qdrant

7/1/2024

Introduced a Universal Query API to consolidate all search types (nearest, fusion, discover, context, recommend) into a single endpoint. Added native support for sparse/dense hybrid search with fusion strategies like RRF, and built-in IDF calculation for sparse vectors. Introduced native support for multivector models like ColBERT via the Query API. Added sparse vector compression. Introduced an Issues API. Added S3 snapshot storage support. Released a new Rust client.

Developing Advanced RAG Systems with Qdrant Hybrid Cloud and LangChain - Qdrant

4/14/2024

This post details the integration of Qdrant Hybrid Cloud with LangChain to build advanced RAG systems. It highlights the flexibility of deploying Qdrant in any environment and leveraging LangChain's framework for vector search. A tutorial is provided for building a RAG-based employee onboarding system using Qdrant Hybrid Cloud, LangChain, and Cohere embeddings.

Iveta Lohovska on Gen AI and Vector Search | Qdrant - Qdrant

4/12/2024

This post features insights from Iveta Lohovska on the importance of trustworthiness and explainability in Generative AI, particularly in high-stakes use cases. It highlights the role of vector databases in providing control and explainability by understanding foundational datasets. The post also touches upon data privacy, on-premises vs. cloud solutions, and the maturity of GenAI models. A specific example of a climate agent is discussed, emphasizing the need for accurate data sources and citations.

Red Hat OpenShift and Qdrant Hybrid Cloud Offer Seamless and Scalable AI - Qdrant

4/11/2024

This post details the integration of Qdrant Hybrid Cloud with Red Hat OpenShift, emphasizing the benefits of this collaboration for deploying and scaling AI workloads. It highlights how Red Hat OpenShift's features like auto-scaling, load balancing, and security controls enhance Qdrant Hybrid Cloud deployments, enabling greater control over data and operational consistency across hybrid cloud environments. A tutorial is provided for building a private chatbot using Qdrant Hybrid Cloud on Red Hat OpenShift with Haystack.

Qdrant Hybrid Cloud and DigitalOcean for Scalable and Secure AI Solutions - Qdrant

4/11/2024

This post details the integration of Qdrant Hybrid Cloud with DigitalOcean, allowing users to deploy Qdrant on DigitalOcean Kubernetes (DOKS) clusters. It highlights the benefits of deploying within one's own infrastructure for data control and flexibility. A tutorial is provided for setting up a RAG application using Qdrant Hybrid Cloud on DigitalOcean with LlamaIndex and Jina AI.

Enhance AI Data Sovereignty with Aleph Alpha and Qdrant Hybrid Cloud - Qdrant

4/11/2024

This post details the integration of Qdrant Hybrid Cloud with Aleph Alpha's foundation models, focusing on enabling data sovereignty and compliance with European standards. It highlights a tutorial for building a region-specific contract management system hosted within Germany for GDPR compliance, demonstrating how to leverage Qdrant's Kubernetes-native design and Aleph Alpha's models for secure, sensitive data processing in AI applications.

STACKIT and Qdrant Hybrid Cloud for Best Data Privacy - Qdrant

4/10/2024

This post announces the integration of Qdrant Hybrid Cloud with STACKIT, allowing for the deployment of a fully managed vector database within STACKIT's German data centers. It emphasizes enhanced data privacy, GDPR compliance, and seamless integration via Kubernetes. A tutorial is provided for building an AI-enabled contract management platform using Qdrant Hybrid Cloud on STACKIT with Aleph Alpha, showcasing multitenancy and custom sharding.

Qdrant and OVHcloud Bring Vector Search to All Enterprises - Qdrant

4/10/2024

This post announces the integration of Qdrant Hybrid Cloud with OVHcloud, allowing for deployment within OVHcloud's infrastructure. It emphasizes data sovereignty, trust, and cost-efficiency through OVHcloud's eco-friendly infrastructure and Qdrant's quantization. A tutorial for building a RAG-based recommendation system on OVHcloud is provided.

New RAG Horizons with Qdrant Hybrid Cloud and LlamaIndex - Qdrant

4/10/2024

This post announces the collaboration between LlamaIndex and Qdrant's new Hybrid Cloud launch. It details how Qdrant Hybrid Cloud, deployed on a Kubernetes-based architecture, allows users to deploy Qdrant in their chosen environment. The integration with LlamaIndex enables the development and scaling of GenAI applications, particularly RAG systems. It highlights the benefits of seamless deployment, open-source compatibility, and advanced search features like hybrid search. A tutorial is provided for building a RAG pipeline for complex PDF documentation using LlamaIndex and Qdrant Hybrid Cloud.

Cutting-Edge GenAI with Jina AI and Qdrant Hybrid Cloud - Qdrant

4/10/2024

This post details the integration of Qdrant Hybrid Cloud with Jina AI's embedding models to enhance Retrieval Augmented Generation (RAG) applications. It highlights the benefits of seamless deployment on any environment, scalable vector search, and cost efficiency through quantization. A tutorial is provided for building a hybrid search application for household appliance manuals using Jina AI embeddings, Qdrant Hybrid Cloud, and the LlamaIndex framework.

Qdrant Hybrid Cloud and Haystack for Enterprise RAG - Qdrant

4/10/2024

This post details the integration of Qdrant Hybrid Cloud with Haystack 2.0, focusing on simplifying the deployment of production-ready Retrieval Augmented Generation (RAG) systems. It highlights the ability to deploy Qdrant in any environment of choice (cloud, on-premise, edge) and its Kubernetes-native design. The integration leverages Haystack's Document Store interface and Hayhooks for creating RESTful APIs from Haystack pipelines. The post also provides a tutorial for building a private RAG chatbot on RedHat OpenShift.

Teaching Vector Databases at Scale - Alfredo Deza | Vector Space Talks - Qdrant

4/9/2024

This post highlights Qdrant's suitability for educational purposes, specifically for teaching vector databases and retrieval augmented generation (RAG). It emphasizes the value of Qdrant's straightforward setup and in-memory implementation for learning experiences. The author, Alfredo Deza, a former Olympic athlete and current Adjunct Professor, uses Qdrant in his university courses, drawing parallels between athletic discipline and consistent learning in AI. The post also touches upon the use of a custom wine dataset for teaching and the broader context of AI learning acceleration in universities.

How to meow on the long tail with Cheshire Cat AI? - Piero and Nicola | Vector Space Talks - Qdrant

4/9/2024

This post details the Cheshire Cat AI framework's native integration with Qdrant as its vector database. It highlights how Cheshire Cat utilizes Qdrant for storing quantized vectors in a hybrid mode to balance memory and performance, and explores Qdrant's alias feature for managing multiple embedders. The framework also leverages Qdrant for managing conversational memories and data.

Talk with YouTube without paying a cent - Francesco Saverio Zuppichini | Vector Space Talks - Qdrant

3/27/2024

This post details the technical implementation of a RAG system that ingests data from YouTube video subtitles. It showcases the use of YouTube DL for subtitle extraction, Hugging Face for generating text embeddings, and Qdrant as the vector database. The author emphasizes running Qdrant and language models locally for privacy and cost-effectiveness, and highlights the importance of metadata in Qdrant for filtering and enhancing RAG query results. The talk also touches upon code organization using functions and the benefits of good documentation.

Integrating Qdrant and LangChain for Advanced Vector Similarity Search - Qdrant

3/12/2024

This post details the integration of Qdrant with LangChain to facilitate Retrieval Augmented Generation (RAG) for GenAI applications. It explains how Qdrant serves as long-term memory for LLMs by storing and retrieving user data efficiently. The post outlines the RAG process, the benefits of using LangChain for simplifying development, and common use cases like chatbots and recommendation systems. It also provides a tutorial outline for building a RAG chatbot and discusses the scalability and production-readiness of the Qdrant-LangChain combination.

IrisAgent and Qdrant: Redefining Customer Support with AI - Qdrant

3/6/2024

This post details the integration of Qdrant into IrisAgent's AI pipeline for customer support automation. It highlights IrisAgent's evaluation criteria for choosing Qdrant, including its open-source nature, performance, ease of use (SDKs, documentation), and maintainability (Rust). The post describes how IrisAgent uses Qdrant for RAG use cases to enhance proprietary AI models for tasks like hallucination detection, similarity detection, and customer intent classification, leading to performance and quality gains. Qdrant is deployed on Google Cloud.

Unlocking AI Potential: Insights from Stanislas Polu - Qdrant

1/26/2024

Stanislas Polu, Co-Founder and Engineer at Dust, discusses their choice of Qdrant for their AI platform. He highlights Qdrant's open-source nature, performance, Rust implementation, and documentation as key decision factors. Polu specifically mentions Qdrant's indexing capabilities on top of vector search for precise data retrieval and its performance benefits for RAG applications. The post also touches on the challenges of enterprise data management and the use of customized AI assistants.

Loading Unstructured.io Data into Qdrant from the Terminal - Qdrant

1/8/2024

This post details the technical steps and commands required to ingest data from Discord channels into Qdrant using the Unstructured.io CLI. It covers setting up prerequisites (Qdrant instance, Discord bot token, Unstructured CLI with specific extras), retrieving data from Discord using `unstructured-ingest discord`, creating a Qdrant collection with specified vector dimensions and distance metrics via a `curl` command, and finally ingesting the structured data into Qdrant using `unstructured-ingest local qdrant`. It highlights configuration options for embedding providers, collection names, API keys, and locations, as well as mentions support for partitioning and chunking.