Agent Caching Optimization

6 posts · Sep 2025 – Jul 2026

What this trail covers

Hex has evolved its agent capabilities by introducing a credit model for Hex Agents, including monthly credit grants per user seat type, and the ability to purchase pooled add-on credits with auto top-ups and spend limits. The consumption is effort-based, with different models offering varying cost-efficiency. Users gain visibility into their credit balance and prompt-level usage, while Admins have access to workspace-wide usage logs, spend limits, and granular control over add-on credit allocation. This post details tactics for credit efficiency, focusing on context management, user prompting, and admin controls to optimize AI usage and cost.

All Hex trails

Jul 2026Caching is all you needThis post details Hex's efforts to optimize AI spend by improving caching strategies. It introduces a cache visualization tool to identify cost drivers, discusses the limitations of average cache hit rates, and outlines three key fixes: improving default user experience with auto-approve and desktop notifications, structuring prompts to keep shared context warm across threads with automated review for regressions, migrating long-running subagents to faster models with time limits, and implementing a better cache route key strategy (bucket key) to reduce mid-run cache misses.
Jul 2026How to Improve Agent Efficiency Through Context EngineeringThis post introduces context engineering as a primary method for improving agent efficiency, arguing it's more impactful than model choice. It details four techniques: retrieval (warehouse metadata, endorsed tables), memory (previous analyses, published notebooks), institutional knowledge (workspace guides), and codified metrics (semantic models). It also discusses context rot, methods for catching it (ground truth testing, trajectory evaluation, conversation-length trends, token efficiency tracking, ongoing evaluation), and the token economics of context at scale, including prompt caching and its cost/performance benefits.
Jul 2026Credits and usage visibility for Hex AgentsIntroduced a credit model for Hex Agents with monthly grants based on user seat type (Professional Editor: 30, Team Editor: 40, Enterprise Editor: 60, Explorer: 10). Enabled purchase of pooled add-on credits with auto top-ups and workspace-wide spend limits. Defined credit consumption based on task effort, model complexity, and context processing. Highlighted the use of subagents and model selection (e.g., Fable 5, Opus 4.7, GPT 5.5, Sonnet 4.6, Kimi 2.7) to optimize cost-efficiency. Provided user-level visibility into credit balance and prompt-level costs, and Admin-level visibility into usage logs, historical data, and spend controls. Integrated credit usage visibility into Context Studio for identifying AI reliance and context curation opportunities.
May 2026Tactics for credit efficiency in HexThis post details practical tactics for optimizing AI credit efficiency within Hex. It elaborates on three key areas: 1) Building out context by adding table metadata, endorsements, and focused guides to reduce agent effort; 2) Guiding users to prompt smarter by referencing existing assets (@-mentions, Collections), starting new Threads instead of long ones, and reserving high-cost models for complex tasks; and 3) Utilizing admin controls like add-on spend limits and per-user credit allocations for planning and managing AI costs. It emphasizes that effort-based pricing scales with agent work, influenced by context gathering, question complexity, and back-and-forth interactions.
Feb 2026Training vs. Inference: What ML Models Actually Cost to RunThis post details the cost differences between ML model training and inference, highlighting that inference costs often exceed training costs due to their recurring nature and scaling with usage. It discusses optimization techniques for inference, including quantization, pruning, knowledge distillation, and memory access optimization (e.g., FlashAttention). The post also touches on deployment strategies (batch vs. real-time) and the importance of continuous retraining and making inference outputs actionable through tools like data apps to manage lifecycle costs effectively.
Advertisement
Oct 2025Bitter lessons building AI productsThis post details the 'bitter lessons' learned in building AI products, emphasizing the shift from clever engineering around model deficiencies to adapting roadmaps based on evolving model capabilities. It highlights past failures with the Notebook Agent and visualization generation due to over-reliance on custom workarounds, and contrasts this with current strategies of early user validation, closely monitoring model capability shifts (e.g., tool calling), and faster project termination when workarounds are needed. The core technical lesson is that adapting to new model capabilities is more effective than building complex systems around current model limitations.
Oct 2025Lessons from Ramp: Running AI-powered data teamsThis post details how Ramp's data team uses Hex's Notebook Agent to accelerate analysis and share insights. Key contributions include: 1. Rapid analysis framework and visualization generation (90 seconds) by providing context (company background, product features, data infrastructure) to the agent. 2. Accelerated exploratory analysis by testing hypotheses directly within the notebook, reducing tool-switching. 3. Synthesizing findings into shareable reports for non-technical audiences (e.g., PMs) by prompting the agent to tailor content. It also emphasizes the importance of a well-documented semantic layer for agent success and discusses the use of evals for ensuring reliability and accuracy of LLM outputs. The post highlights that both technical and quasi-technical users are benefiting from the agent.
Sep 2025AI: because your team has better things to do than writing boilerplate SQLThis post details the data team's pragmatic approach to using AI, specifically the Notebook Agent, to improve efficiency and unlock insights. It outlines principles for applying AI, such as testing liberally, time-boxing experiments, keeping humans in the loop, and ensuring work influences outcomes. The post provides concrete examples of tasks AI has accelerated (drafting dbt documentation, creating semantic layers, bootstrapping analyses) and where it's not yet ready (complex data model changes, bug identification). It also discusses how AI-driven insights can lead to business impact, citing examples like analyzing project metadata and building forecast models, while cautioning against "trivia insights" and emphasizing the need for human interpretation and decision-maker alignment. The post also touches on the iterative nature of AI development, noting that current limitations may be overcome quickly.