TechPrufer
Get started
Databricks
Databricks logoDatabricks · Feature trail

Agent Performance and Cost Optimization

11 posts · Jul 2026 – Oct 2026

What this trail covers

This post details how Databricks' Unity AI Gateway is being used to integrate Kimi K3 from Moonshot AI, enabling users to leverage advanced AI models within the Databricks ecosystem. It highlights the expansion of governable AI models and services through the gateway, focusing on agent performance and cost optimization. The post also details how Databricks' Unity AI Gateway is being used to manage and control AI spend, ensuring efficient utilization of AI resources. This post further details the integration of AI Search and Lakebase Postgres for e-commerce recommendations, focusing on real-time data ingestion, low-latency serving paths, and handling cold-start problems. It also covers model lifecycle management and continuous improvement for personalization engines.

All Databricks trails
  1. Real-Time Retail Intelligence: Building E-Commerce Recommendations with Lakebase and AI Search on DatabricksThis post details the implementation of an e-commerce recommendation system using Databricks. It covers real-time data ingestion from clickstream data into the Lakehouse in seconds, two distinct low-latency serving paths for recommendations (one leveraging Lakebase Postgres for immediate access and another for more complex model-based recommendations), strategies for handling the cold-start problem with techniques like content-based filtering and hybrid approaches, and the model lifecycle management for continuous improvement of recommendation models. The system integrates AI Search for enhanced product discovery and leverages Databricks' capabilities for scalable data processing and model serving.
  2. How to scale agentic applications without creating AI sprawlThis post addresses the challenge of scaling agentic applications without introducing AI sprawl by focusing on three key principles: sharing context, enabling choice without fragmentation, and ensuring control follows action. It advocates for a unified infrastructure that supports composition, allowing for the development of scalable and manageable agentic systems. The post emphasizes building for choice, context, and control to prevent the proliferation of unmanaged AI components.
  3. How Databricks rolls out frontier models to 12,000 employees on Day 1This post details the engineering challenges and solutions for rolling out frontier AI models to 12,000 employees on Day 1. It addresses the discrepancy between marketed 'frontier' models and actual performance/cost by implementing a tiered rollout strategy based on model performance and cost metrics. The system uses a combination of internal benchmarking, cost analysis, and a phased rollout to ensure employees have access to the best available models without incurring excessive costs or encountering underperforming models. Key technical decisions include establishing a model evaluation framework, defining cost thresholds, and implementing a mechanism to dynamically switch between models based on these criteria.
  4. How Databricks rolls out frontier models to 14,000 employees on Day 1This post details the rollout strategy for frontier AI models to 14,000 employees on Day 1 at Databricks. It describes the infrastructure and operational challenges of deploying new, potentially unstable models to a large internal user base, focusing on rapid iteration, feedback loops, and ensuring a smooth user experience. Key technical aspects include the use of a canary deployment system, automated rollback mechanisms, and a robust observability stack to monitor model performance and user impact in real-time. The post highlights the trade-offs between speed of deployment and stability, and the engineering effort required to balance these.
  5. Health Plans: Your BI Tells You MLR Moved. Can Your AI Tell You Why?This post details how Databricks' Unity AI Gateway is being enhanced with smart routing to match frontier quality AI models with over 30% lower cost per task. This builds upon existing efforts in agent performance and cost optimization by introducing intelligent routing mechanisms that dynamically select the most cost-effective model while maintaining high quality, likely involving sophisticated cost-benefit analysis and model performance monitoring. The post also details how health plans can leverage AI to understand the 'why' behind MLR movements, moving beyond simple BI reporting to actionable AI-driven insights.
  7. Evaluation-First AI Agents: How Zepto Scales Customer Support on Databricks and MLflowThis post details the implementation of evaluation-first AI agents to scale customer support on Databricks and MLflow. It highlights Zepto's approach to agent development, focusing on rigorous evaluation to ensure performance and cost-effectiveness. The post likely describes the architecture of these agents, the evaluation methodologies employed, and how MLflow is used for tracking experiments and managing agent deployments. It also touches upon the integration with Databricks for scaling these agents, implying a focus on production readiness and operational efficiency.
  8. How Trackunit turns construction data into decisions with AIThis post details the implementation of a real-time fraud prevention system for government benefits, focusing on the deployment of 'agents' for production lines. It covers the architecture and technical considerations for processing transactions and identifying fraudulent activities as they occur, enabling immediate action to mitigate losses. The system likely involves stream processing, anomaly detection, and integration with existing benefit disbursement systems.
  9. Designing effective Genie Agents from a single promptThis post details the design and implementation of effective Genie Agents from a single prompt, focusing on prompt engineering techniques to improve agent performance and reduce costs. It explores how to translate a single prompt into a robust agent capable of complex tasks, likely involving techniques for prompt decomposition, context management, and output validation.
  10. Smart Routing in Unity AI Gateway: Match frontier quality with 30%+ lower cost per taskThis post details the enhancement of Databricks' Unity AI Gateway with smart routing to optimize AI model selection for cost and performance. It introduces intelligent routing mechanisms that dynamically choose the most cost-effective model while maintaining high quality, likely involving sophisticated cost-benefit analysis and model performance monitoring. This aims to achieve over 30% lower cost per task for frontier quality AI models.
  11. Managing AI Coding Costs at ScaleThis post details strategies for managing AI coding costs at scale, focusing on techniques such as prompt engineering, model selection, and efficient execution. It highlights how Databricks' Unity AI Gateway facilitates cost optimization by intelligently routing tasks to the most cost-effective frontier quality AI models, achieving over 30% cost reduction per task. The post also touches upon the use of AI agents for coding tasks, implying a focus on developer productivity and resource management within AI-driven development workflows.
  12. What are Agentic Workflows?This post introduces the concept of 'Agentic Workflows' and details how Databricks is enabling them. It explains that agentic workflows involve AI agents that can autonomously plan, execute, and iterate on tasks. The post highlights the technical aspects of building these workflows, including the use of tools, planning mechanisms, and execution environments. It also touches upon the challenges and opportunities in deploying and managing these complex AI systems, emphasizing the need for robust infrastructure and orchestration capabilities. The post specifically mentions the integration of AI agents into production lines for real-time fraud prevention and the development of a self-serve infrastructure vending machine for the agentic era.
  13. Why A Frontier Data Agent Outperforms General Coding Agents in Quality and CostThis post introduces a frontier data agent that outperforms general coding agents in quality and cost for specific data tasks. It details how this agent leverages specialized knowledge and reasoning capabilities to achieve superior results compared to general-purpose coding agents. The post highlights the technical trade-offs in agent design, focusing on how domain-specific agents can optimize for both performance and cost-efficiency in data-intensive applications, contrasting their capabilities with broader coding agents.