TechPrufer
Get started
Qdrant
Qdrant logoQdrant · Feature trail

On-Device Vector Search

4 posts · Jul 2025 – Jun 2026

What this trail covers

Qdrant Edge enables on-device vector search for real-time anomaly detection in video streams. This system leverages a split architecture with an immutable HNSW shard for baseline normal activity and a mutable shard for live writes, allowing for sub-millisecond kNN lookups with no network dependency and full offline resilience. Video embeddings are provided by Twelve Labs Marengo 3.0, and NVIDIA Metropolis VSS orchestrates GPU-accelerated ingestion on Vultr Cloud GPUs. The system escalates anomalies detected locally to a cloud-based Qdrant instance for further analysis and model retraining. This capability is being expanded for robotics, mobile devices, point-of-sale systems, and IoT agents, requiring minimal runtime dependencies and strict latency bounds.

All Qdrant trails
  1. Memory at the Edge: On-Device Vector Search with Qdrant Edge - QdrantThis post introduces Qdrant Edge, a new deployment mode for Qdrant that runs as an in-process embedded library. It details the architecture of Qdrant Edge, which allows for fully offline, sub-millisecond vector search on local disk. The post highlights the use of Qdrant Edge in a robot memory demo, showcasing its ability to capture, embed, and search observations locally using dense and sparse vectors with payload filters. It also outlines three patterns for on-device vector search: Robotic Memory, Edge Anomaly Triage, and Private Device Memory, emphasizing the 'local first, cloud when needed, sync between' paradigm. The primary technical tradeoff discussed is the shift from cloud-first retrieval to edge-based retrieval to address latency, connectivity, cost, privacy, and isolation concerns.
  2. Video Anomaly Detection From Edge to Cloud With Qdrant - QdrantThis post details the implementation of a real-time video anomaly detection system using Qdrant Edge. It introduces a two-shard architecture for Qdrant Edge (immutable HNSW for baseline, mutable for live writes) enabling sub-millisecond kNN lookups with offline resilience. It also describes the integration with Twelve Labs Marengo 3.0 for video embeddings and NVIDIA Metropolis VSS for GPU-accelerated ingestion, and outlines an edge-to-cloud escalation pipeline to optimize processing costs.
  3. How Tavus used Qdrant Edge to create conversational AI - QdrantThis post details how Tavus implemented per-conversation edge collections using Qdrant Edge to achieve sub-second conversational AI. The core technical contribution is the architectural decision to colocate Qdrant instances with conversational workers, eliminating network hops for embedding and ANN lookups. This design choice prioritized retrieval quality and multimodal accuracy over micro-optimizations like quantization, enabling retrieval on every utterance and speculative execution. The result was a reduction in retrieval latency to 20-25ms at the edge, contributing to an end-to-end utterance-to-utterance timing of 500-600ms, which feels natural to users. The post highlights the business impact of this architectural choice, including simplified onboarding for customers and improved developer velocity.
  4. Qdrant Edge: Vector Search for Embedded AI - QdrantThis post announces Qdrant Edge, a lightweight, embedded vector search engine designed to run on local hardware with limited resources. It operates as an in-process library, not a service, with synchronous operations for search and indexing. Key design targets include robotics, mobile devices, point-of-sale systems, and IoT agents, emphasizing low-latency retrieval, multimodal inputs, and bandwidth-independent operation. It retains Qdrant's core search and filtering capabilities but is re-architected for minimal footprint and multitenancy-aware deployments on edge devices.