Not all enterprise context is created equal: Glean results are preferred ~2× more than ChatGPT and 1.6× more than Claude

This post details an evaluation comparing Glean's enterprise context layer against OpenAI ChatGPT's company knowledge and Anthropic Claude's enterprise search. It found that human graders preferred Glean's answers 1.9x more often than ChatGPT's and 1.6x more often than Claude's on correctness. The post explains the challenges of context rot in LLMs and the importance of a robust enterprise context layer for agentic work. It outlines the evaluation design, including the queryset, dataset construction (using Glean's internal deployment as a proxy), and grading methodology (blind, randomized evaluation on correctness and completeness). The results show Glean's strengths in finding canonical documents, understanding jargon, handling ambiguity, and answering technical queries, attributed to its semantic search models and Enterprise Graph. It also contrasts Claude's 'deep' sequential tool-calling strategy with ChatGPT's 'wide' parallel search strategy, highlighting potential inefficiencies and inaccuracies in both. The post emphasizes the persistence of indexes and knowledge graphs due to their accuracy and avoidance of brute-force compute.