Introducing Exa's People Search Benchmarks

Introduced a new benchmark for people search, including a role-based people search dataset (1,400 queries) and an evaluation harness. Developed a retrieval system for people search, indexing over 1 billion people using hybrid retrieval (fine-tuned Exa embeddings + Exa search). The ingestion pipeline is designed to handle over 50 million updates per week. Analyzed 10,000 historical people queries to identify dominant usage patterns (role-based search, skill/role-based discovery, specific individual lookup). Generated role-based queries using a structured taxonomy and sampled executives/founders across different company market cap tiers. Implemented evaluation methodology using recall@k, NDCG for targeted lookups, and an LLM judge for role-based discovery.