Exa Feature Trails

Exa has launched a dedicated search capability for academic publications, indexing approximately 350 million publications and 30 million authors. This system allows for semantic search using natural language, even with vague or incomplete queries, and has demonstrated superior recall and latency compared to existing academic search engines. The ingestion pipeline includes OCR and document parsing for PDFs, combined with metadata such as authors, citations, and publication history. The system also supports semantic search using transformers end-to-end, enabling filtering by meaning rather than keywords. The recent announcement of Exa as the AI Search Engine with Semantic Search Technology further emphasizes this capability, highlighting its ability to organize the world's knowledge by filtering information based on meaning, contrasting with keyword-based search engines. The new 'highlights' feature allows for instant extraction of webpage content using customizable embedding models, chunking and embedding full webpages with a paragraph prediction model, enabling live customization of highlight length, quantity per page, and secondary query specification.

Exa has launched exa-code, a web-scale context tool for coding agents. It leverages Exa's search engine and prioritizes code examples to provide highly relevant and dense context, typically a few hundred tokens. The system hybrid searches over 1B+ webpages, extracts and reranks code examples using an ensemble method, and returns either concise code examples or full documentation pages. Evaluations show exa-code significantly reduces LLM hallucinations in coding tasks by providing optimized code example recall from a dedicated index sourced from GitHub and Exa's web index, using code-specific retrieval models. The exa-code MCP is available via Smithery and Exa's documentation.

Exa has developed Canon, a search pipeline orchestrator designed to manage the increasing complexity of its search engine. Canon represents search pipelines as Directed Acyclic Graphs (DAGs), enabling automatic parallelism, durable execution, introspectability, and decoupling of definition from execution. The runtime evaluates these DAGs using a pull-based system, maximizing compute efficiency through automatic cancellation propagation and memoization, and providing comprehensive observability. This post details the optimization of the Exa Highlights real-time embedding service, migrating from Python to Rust to achieve 4x throughput by overcoming GIL contention and improving parallelism for CPU-bound text pre-processing and GPU-bound model inference. It also addresses challenges with CUDA memory fragmentation and thread safety when using parallel iterators with GPUs.

Exa has significantly optimized its BM25 index, reducing memory overhead by over 50% across billions of documents. This was achieved through a combination of algorithmic improvements like smart candidate selection and dynamic pruning, and low-level memory structure optimizations. Key techniques include frequency-based organization, variable-length + delta encoding for document IDs, Zstd compression, consolidated buffers, singleton optimization, and further consolidation of buffers into a single continuous vector. These optimizations not only reduce memory consumption and operational costs but have also led to a 10% improvement in average query latency due to more predictable memory access patterns.