Right-sizing intelligence: Glean saves 81% on token costs and is preferred 78% of the time versus Claude Cowork

This post details Glean's approach to optimizing AI chat latency and cost by implementing a Pareto frontier analysis to map cost and performance tradeoffs across various models. It introduces Glean's auto-routing mechanism, which matches tasks to models and their reasoning levels based on effort and capability required, using a specialist model (Glean Waldo) for reasoning level determination. The post also presents benchmark analysis showing Glean's 81% cost savings and 78% preference rate over Claude Cowork, attributing this to architectural decisions like a unified enterprise context and an efficient harness design that minimizes token usage. It highlights how Glean's preferred responses reduce 'botsitting' by providing more accurate, complete, and grounded outputs.
More from the AI Chat Latency Optimization trail
- A complete guide to retrieval augmented generation vs fine-tuning
- How KV caches impact time to first token for LLMs
- How input token count impacts the latency of AI chat tools
- LLM choice in Assistant: A better agentic coworker sourcing deep proficiencies from your models of choice
- 5 metrics to measure AI-generated answers' decision-making impact