TechPrufer
Sign inGet started

Blogs›Grab›AI-Assisted Analytics

AI-Assisted Analytics

AI-Assisted Analytics

5
posts
2026

This post details the development of Grab Bench, a configurable evaluation harness for AI systems on Grab-shaped production work. It addresses the challenge of evaluating AI models beyond superficial correctness by focusing on metric faithfulness, tool-parameter discipline, evidence grounding, and safety boundaries. Grab Bench uses task plugins to run models through specific scenarios, records row-level failures, and employs deterministic scorers or LLM judges. Design choices emphasize making cases safe with synthetic data, scoring contracts over confidence, and making shortcuts visible. The system includes teaching artifacts for learning and certification artifacts for generalization testing, with gates to ensure reproducibility and catch regressions. Future work aims to evaluate pipelines and make packages more reusable.

2026

Grab Bench: Evaluating AI on Grab-shaped production work

8/1/2026

Introduced Grab Bench, an evaluation harness for AI systems on production-like tasks. Designed to evaluate AI models on specific contracts like metric faithfulness, tool parameter discipline, and evidence grounding, rather than just general confidence. Implemented a system with task plugins, deterministic scorers/LLM judges, and a focus on making failure modes visible. Developed a split between teaching and certification artifacts for reproducibility and generalization testing. Established gates for evaluating the harness itself.

How AI is transforming analytics at Grab

7/30/2026

Introduces a five-level framework for AI autonomy in analytics, detailing the progression from AI-assisted to end-to-end autonomous analytical loops. Describes the development and implementation of Spartan, an end-to-end agentic analytics workflow, and Scarlet, an agent for automated root-cause analysis and pipeline repair. Highlights the importance of ContextIQ for maintaining up-to-date analytical context and the development of BriX as a platform for creating specialized AI-driven analytics interfaces. Details specific AI skills and analysis frameworks used within these systems, and discusses the evolution of analyst roles towards problem framing and strategic decision-making.

Palana (Part 1): Why Grab built a secure platform for autonomous AI Agents

7/24/2026

This post introduces Palana, Grab's Kubernetes-native platform for running autonomous AI agents securely. It details the platform's design principles including isolation as the unit of trust, never giving credentials directly to the agent, making egress a control point via Envoy and OPA, and ensuring the control plane remains outside the agent. It also highlights the use of Kubernetes primitives for managing agent lifecycles and resources.

Palana (Part 2): Architecting isolation, identity, and auditability for AI agents

7/24/2026

This post dives into the technical architecture of Palana, Grab's secure execution platform for AI agents. It details the implementation of isolation, identity, and auditability through Kubernetes namespaces, RBAC, network policies (Cilium), and proxy-based access control (Traefik, OAuth2-Proxy, OPA). It also explains the secrets management strategy using Vault, LLM access via a dedicated proxy wrapper, and the observability mechanisms including structured logging and an idle-shutdown reaper. The post highlights lessons learned regarding platform-level security, user experience, credential handling, namespace compounding, and lifecycle management for long-running agents.

Agent platform (Part 1): How we help Grab build and run AI agents at scale

7/10/2026

This post details the initial architecture of Grab's AI support bot, which served as the precursor to LLM-Kit. It describes the reasoning plane (single-agent loop) and the tool plane (integrating Glean, Kibana, GitLab, Slack). It also outlines the key pain points encountered during scaling: lack of robust evaluation strategies, the need for fast model and provider switching, the critical importance of observability, and the significant effort required for production plumbing (auth, secrets, config, etc.). These pain points directly led to the development of the LLM-Kit framework, which aims to standardize these surrounding components.