TechPrufer
Get started
Exa
Exa logoExa · Feature trail

Search Pipeline Orchestration

13 posts · Feb 2024 – Apr 2026

What this trail covers

Exa has developed Canon, a search pipeline orchestrator designed to manage the increasing complexity of its search engine. Canon represents search pipelines as Directed Acyclic Graphs (DAGs), enabling automatic parallelism, durable execution, introspectability, and decoupling of definition from execution. The runtime evaluates these DAGs using a pull-based system, maximizing compute efficiency through automatic cancellation propagation and memoization, and providing comprehensive observability. This post details the runtime's pull-based execution model, memoization for diamond dependencies, and automatic cancellation propagation. It also elaborates on the observability benefits, including tracing individual query paths and decisions made by the search engine, and how this enables agents to inspect graphs and identify root causes of unexpected behavior.

All Exa trails
  1. Exa HighlightsIntroduced significant improvements to Exa's 'highlights' extraction method, a core component of the search pipeline. The new highlights model extracts relevant excerpts from web pages with higher quality and drastically reduced token usage (~94% fewer tokens on some evals), leading to cost and latency benefits. This improvement is particularly effective for long-context documents and technical coding docs. The highlights are now run for every request (not cached) and complete in under 100ms, serving as the retrieval substrate for agentic products.
  2. Composing a Search EngineThis post details the runtime's pull-based execution model, explaining how it maximizes compute efficiency through automatic cancellation propagation and memoization for diamond dependencies. It also elaborates on the observability benefits, including tracing individual query paths and decisions made by the search engine, and how this enables agents to inspect graphs and identify root causes of unexpected behavior.
  3. Introducing Exa Instant: the fastest search engine in the worldThis post introduces Exa Instant, a new sub-200ms search engine. It details the engineering effort required to achieve this low latency, focusing on optimizing the search and retrieval stack. The post contrasts Exa Instant's performance with Google SERP-based search APIs and highlights its value for AI agents and low-latency products. It also mentions the use of SealQA queries and random word generation to avoid caching during benchmarking.
  4. exa-d: Data Framework to Process the WebThis post introduces exa-d, a new data processing framework for Exa. exa-d is designed to handle the ingestion and processing of web data at scale, addressing challenges related to numerous outputs per page, heterogeneous content, varying update frequencies, and sheer volume. Key features include typed columns and declarative dependencies, surgical updates and full rebuilds for efficient data modification, and efficient parallel execution. It uses Lance for storage on S3, with a storage model that supports writing or deleting a single column for a specific fragment without rewriting the entire fragment. The framework represents data transformations as a dependency graph and leverages this graph for execution order and to ensure column definitions act as contracts.
  5. Zero Data Retention across Exa Search ProductsThis post announces the extension of Zero Data Retention (ZDR) to Exa's /answer and /research endpoints, in addition to the existing ZDR search capability. It highlights the technical requirement of having an independent search engine to achieve ZDR, contrasting it with providers that rely on external search engines like Google. The core technical contribution is the architectural decision and implementation to ensure that user query data is deleted after each search, enabling privacy-sensitive use cases.
  7. The World's Fastest Search APIIntroduced Exa Fast, a new, streamlined search API with p50 latency below 425ms. This was achieved by building the entire search stack from scratch, including crawling, search models, and a proprietary vector database, enabling fine-grained control over latency optimization. The post highlights the importance of low-latency search for AI systems and agentic workflows, contrasting it with wrapper-based APIs.
  8. The Exacluster: Powering Our Neural Network Search EngineThis post details the infrastructure powering Exa's neural network search engine, including a large GPU cluster (Exacluster) built with 144 H200 GPUs and 3,456 CPUs, managed by Pulumi for IaC, Ansible/Kubespray for Kubernetes deployment, NVIDIA operators for hardware management, Alluxio for distributed caching of local NVMe storage, and Flyte for workflow orchestration. This setup enables rapid training and deployment of large-scale ML models.
  9. How We're Building the Next Generation of Search with Semantic Search TechnologyThis post introduces the core infrastructure for Exa's next-generation search engine, detailing the crawling process, HTML parsing, S3 storage, and the critical step of preprocessing documents into neural embeddings using specialized transformer models. It highlights the "Bitter Lesson" philosophy by emphasizing the use of significant compute resources, including a 144 H200 GPU "Exacluster," for training these embedding models. The post also outlines two distinct product offerings, Exa Search (low-latency general search) and Exa Websets (high-compute, complex query search), with a future goal of unification. The architecture diagram illustrates the flow from crawling to serving queries via a custom vector database.
  10. Websets vs. Google: The Most Powerful AI Research AgentIntroduces "Websets", a new search product that significantly outperforms Google and OpenAI's Deep Research on complex queries. Details the evaluation methodology, including the creation of a custom benchmark and the use of GPT-4o for result grading. Highlights the technical approach of deploying agents with a custom embedding-based search algorithm and the impact of compute resources on retrieval quality.
  11. Web Search API Evals: Exa's Neural Network Search Engine vs. The CompetitionThis post details the evaluation of Exa's neural network search engine against competitors using benchmarks like SimpleQA and MSMARCO, demonstrating its state-of-the-art performance in RAG and classic search applications. The evaluation methodology highlights the use of LLMs for query generation and answer summarization, and LLM-as-Judge for result grading, showcasing the effectiveness of Exa's retrieval system for LLM integration.
  12. How We Built a Web-Scale Vector Database for Our Neural Network Search EngineThis post details the technical implementation of Exa's web-scale vector database. Key contributions include: 1. Matryoshka embeddings to reduce vector dimensionality by 20x. 2. Binary quantization to further reduce memory by 16x. 3. A novel hybrid search approach using uncompressed query embeddings with binary document embeddings and a hyper-optimized dot product calculation via precomputed lookup tables, reducing computations by 1/4. 4. Loading lookup tables into CPU registers for faster access. 5. Clustering documents into 100,000 groups for targeted searching, yielding a ~1000x throughput improvement. The post also describes the integration of metadata filtering using inverted indexes and reranking with uncompressed data to maintain recall.
  13. Scaling Our Server for Semantic Search Technology HighlightsThis post details the optimization of the Exa Highlights real-time embedding service, migrating from Python to Rust to achieve 4x throughput by overcoming GIL contention and improving parallelism for CPU-bound text pre-processing and GPU-bound model inference. It also addresses challenges with CUDA memory fragmentation and thread safety when using parallel iterators with GPUs.