Search Pipeline Orchestration

4/22/2026

Introduced significant improvements to Exa's 'highlights' extraction method, a core component of the search pipeline. The new highlights model extracts relevant excerpts from web pages with higher quality and drastically reduced token usage (~94% fewer tokens on some evals), leading to cost and latency benefits. This improvement is particularly effective for long-context documents and technical coding docs. The highlights are now run for every request (not cached) and complete in under 100ms, serving as the retrieval substrate for agentic products.

4/17/2026

This post introduces Canon, a new search pipeline orchestrator built at Exa. It details the motivation for its creation, stemming from the complexity of modern search engines and the challenges of serving AI agents. The core technical contribution is the representation of search pipelines as DAGs, with explanations of the benefits this provides (automatic parallelism, durable execution, introspectability, decoupled definition/execution). It also describes the pull-based runtime architecture, its role in maximizing compute efficiency and providing observability, and the importance of nodes interacting with the runtime through a uniform interface.

2/12/2026

This post introduces Exa Instant, a new sub-200ms search engine. It details the engineering effort required to achieve this low latency, focusing on optimizing the search and retrieval stack. The post contrasts Exa Instant's performance with Google SERP-based search APIs and highlights its value for AI agents and low-latency products. It also mentions the use of SealQA queries and random word generation to avoid caching during benchmarking.

1/13/2026

This post introduces exa-d, a new data processing framework for Exa. exa-d is designed to handle the ingestion and processing of web data at scale, addressing challenges related to numerous outputs per page, heterogeneous content, varying update frequencies, and sheer volume. Key features include typed columns and declarative dependencies, surgical updates and full rebuilds for efficient data modification, and efficient parallel execution. It uses Lance for storage on S3, with a storage model that supports writing or deleting a single column for a specific fragment without rewriting the entire fragment. The framework represents data transformations as a dependency graph and leverages this graph for execution order and to ensure column definitions act as contracts.

8/18/2025

This post announces the extension of Zero Data Retention (ZDR) to Exa's /answer and /research endpoints, in addition to the existing ZDR search capability. It highlights the technical requirement of having an independent search engine to achieve ZDR, contrasting it with providers that rely on external search engines like Google. The core technical contribution is the architectural decision and implementation to ensure that user query data is deleted after each search, enabling privacy-sensitive use cases.

7/29/2025

Introduced Exa Fast, a new, streamlined search API with p50 latency below 425ms. This was achieved by building the entire search stack from scratch, including crawling, search models, and a proprietary vector database, enabling fine-grained control over latency optimization. The post highlights the importance of low-latency search for AI systems and agentic workflows, contrasting it with wrapper-based APIs.

5/15/2025

This post details the infrastructure powering Exa's neural network search engine, including a large GPU cluster (Exacluster) built with 144 H200 GPUs and 3,456 CPUs, managed by Pulumi for IaC, Ansible/Kubespray for Kubernetes deployment, NVIDIA operators for hardware management, Alluxio for distributed caching of local NVMe storage, and Flyte for workflow orchestration. This setup enables rapid training and deployment of large-scale ML models.

3/11/2025

This post introduces the core infrastructure for Exa's next-generation search engine, detailing the crawling process, HTML parsing, S3 storage, and the critical step of preprocessing documents into neural embeddings using specialized transformer models. It highlights the "Bitter Lesson" philosophy by emphasizing the use of significant compute resources, including a 144 H200 GPU "Exacluster," for training these embedding models. The post also outlines two distinct product offerings, Exa Search (low-latency general search) and Exa Websets (high-compute, complex query search), with a future goal of unification. The architecture diagram illustrates the flow from crawling to serving queries via a custom vector database.

2/19/2025

Introduces "Websets", a new search product that significantly outperforms Google and OpenAI's Deep Research on complex queries. Details the evaluation methodology, including the creation of a custom benchmark and the use of GPT-4o for result grading. Highlights the technical approach of deploying agents with a custom embedding-based search algorithm and the impact of compute resources on retrieval quality.

1/24/2025

This post details the evaluation of Exa's neural network search engine against competitors using benchmarks like SimpleQA and MSMARCO, demonstrating its state-of-the-art performance in RAG and classic search applications. The evaluation methodology highlights the use of LLMs for query generation and answer summarization, and LLM-as-Judge for result grading, showcasing the effectiveness of Exa's retrieval system for LLM integration.

12/17/2024

This post details the technical implementation of Exa's web-scale vector database. Key contributions include: 1. Matryoshka embeddings to reduce vector dimensionality by 20x. 2. Binary quantization to further reduce memory by 16x. 3. A novel hybrid search approach using uncompressed query embeddings with binary document embeddings and a hyper-optimized dot product calculation via precomputed lookup tables, reducing computations by 1/4. 4. Loading lookup tables into CPU registers for faster access. 5. Clustering documents into 100,000 groups for targeted searching, yielding a ~1000x throughput improvement. The post also describes the integration of metadata filtering using inverted indexes and reranking with uncompressed data to maintain recall.

2/21/2024

This post details the optimization of the Exa Highlights real-time embedding service, migrating from Python to Rust to achieve 4x throughput by overcoming GIL contention and improving parallelism for CPU-bound text pre-processing and GPU-bound model inference. It also addresses challenges with CUDA memory fragmentation and thread safety when using parallel iterators with GPUs.