TechPrufer
Get started
Grab
Grab logoGrab · Feature trail

AI Agent Framework

3 posts · Nov 2024 – Jul 2026

What this trail covers

Grab's AI Gateway provides a centralized, unified interface for accessing multiple GenAI providers (OpenAI, Azure, AWS Bedrock, Google VertexAI, etc.). It streamlines access, enables experimentation with various LLMs, optimizes cost-efficiency through shared capacity, and enforces auditing and platformization benefits like monitoring, cost attribution, and rate limiting. The architecture acts as a reverse proxy, handling authentication, authorization, and rate limiting, while also supporting div. The LLM-Kit is a comprehensive framework designed to supercharge the setup of production-ready LLM applications, providing a pre-configured structure, integrated tech stack, built-in observability, efficient config and secret management, authentication, API documentation, Redis & vector database integration, a deployment pipeline, and seamless integration with LangSmith for evaluations. It has onboarded hundreds of GenAI applications and is the de facto choice for developers, saving an estimated 1.5 weeks per team before feature development.

All Grab trails
  1. Grab Bench: Evaluating AI on Grab-shaped production workThis post introduces Grab Bench, an evaluation harness for AI systems. It details the design choices behind the harness, focusing on creating safe evaluation cases using synthetic data, scoring based on task contracts rather than just confidence, and making potential shortcuts visible. It also describes the use of teaching and certification artifacts for reproducibility and generalization, and outlines future plans for pipeline-like evaluation surfaces and broader reusability.
  2. Palana (Part 1): Why Grab built a secure platform for autonomous AI AgentsThis post introduces Palana, a Kubernetes-native platform for securely running autonomous AI agents. Palana provides isolation via per-agent namespaces, controlled egress through Envoy and OPA, proxy-mediated secrets for credential management, and external control plane mechanisms like kill switches. It addresses the challenges of running agents with capabilities like network access, credentials, and memory without compromising security or operational visibility. The platform supports various agent types, including remote development environments and Slack automation, and is built on Kubernetes primitives.
  3. Palana (Part 2): Architecting isolation, identity, and auditability for AI agentsThis post details the architecture of Palana, Grab's Kubernetes-native secure execution platform for autonomous AI agents. It covers the agent lifecycle, identity management (OIDC, RBAC, SPIFFE/SPIRE integration), secrets management (Vault with least privilege), LLM access via a LiteLLM proxy wrapper, and network access controls (Kubernetes NetworkPolicy, OPA-based proxy policies). It also discusses observability, idle shutdown, and lessons learned, emphasizing platform-layer security controls, user experience, separating credential reading from execution, namespace boundaries, and lifecycle management for long-running agents. Future patterns include supervisor systems, Slack-native agents, remote development environments, agent swarms, and operational agents.
  4. Agent platform (Part 1): How we help Grab build and run AI agents at scaleThis post details the evolution of Grab's AI agent development from a single support bot to the LLM-Kit framework. It highlights the initial pain points encountered with the support bot, such as lack of evaluation strategies, slow model switching, poor observability, and the overhead of setting up production infrastructure. LLM-Kit was developed to address these issues by providing standardized scaffolding and integrations for building AI agents. Key contributions include a template for FastAPI services with pre-assembled production wrappers, built-in evaluation routes (ROUGE, BLEU, LLM-as-judge), and solved problems around configuration, secrets management, model access via the GrabGPT Gateway, and observability using OpenTelemetry. The framework aims to significantly reduce the time and effort required to ship AI agents to production.
  5. From firefighting to building: How AI agents restored our team’s core productivityThis post details the implementation of a multi-agent AI system for the Analytics Data Warehouse (ADW) team to automate responses to common data-related queries and enhancement requests. The system leverages FastAPI and LangGraph for agent orchestration, with specialized agents (Classifier, Data Agent, Code Search Agent, On-call Agent, Enhancement Agent, Summarizer) collaborating to handle tasks. The Enhancement Agent semi-automates code changes for pipeline modifications, while the investigation pathway uses a coordinated team of agents to trace data lineage, query data, check production systems, and summarize findings. The architecture decouples the LLM 'brain' from specialized agents acting as 'hands', improving maintainability and accuracy.
  7. From deployment slop to production reality: How BriX bridges the gap with enterprise-grade AI infrastructureThis post introduces BriX, a platform that addresses the 'deployment slop' of AI prototypes by providing an enterprise-grade AI infrastructure layer. BriX enables domain experts to deploy AI tools rapidly by handling security, scaling, data connections, monitoring, and audit trails through configuration rather than extensive coding. Key capabilities include model agnosticism (switching between models like Claude and GPT), secure enterprise data connections via Model Context Protocols (MCPs), and ensuring consistent behavior through centralized system prompts and context files. The BriX architecture is described as a synchronous streaming system with a React frontend using SSE, a FastAPI backend, and LangGraph for AI workflow orchestration, incorporating identity propagation and full context processing for data retrieval.
  8. SpellVault’s evolution: Beyond LLM apps, towards the agentic futureThis post details the evolution of Grab's SpellVault platform from a no-code LLM app builder to an agentic framework. Key technical contributions include: 1. Transitioning from a legacy executor system to a graph-based executor supporting branching, looping, and modularity, enabling ReAct agent behaviors. 2. Decoupling executor and prompt engineering components to allow for multiple execution pathways, including generic Deep Research and sophisticated internal workflows. 3. Consolidating scattered capabilities (Python execution, internal repository search) into "Native Tools" and rebranding user-created plugins as "Community Built Tools" under a unified "Tools" abstraction. This enables AI agents to dynamically invoke these tools for reasoning and action, shifting the platform towards more natural, discoverable, and extensible agentic behavior.
  9. User foundation models for GrabThis post introduces Grab's user foundation model, built on a transformer architecture, for generating user embeddings. It addresses the challenges of unifying tabular and time-series data, handling diverse modalities (text, numerical, IDs, locations), generalizing across tasks, and scaling to massive entity vocabularies. The model uses a novel `key:value` tokenization strategy for unified input and employs specialized encoders for different data types. It can be used to extract user embeddings for downstream tasks or fine-tuned for direct predictions, improving personalization and performance across various Grab verticals.
  10. From failure to success: The birth of GrabGPT, Grab’s internal ChatGPTThis post details the creation of GrabGPT, an internal ChatGPT-like tool. It describes an initial failed attempt to build a support chatbot using chatbot-ui and GPT-3.5-turbo, which was limited by token constraints and embedding search limitations. The pivot led to the development of GrabGPT, an internal tool that leverages Grab's model-serving platform (Catwalk) and supports multiple LLM providers (OpenAI, Claude, Gemini). The success of GrabGPT is attributed to its private route for data security, global accessibility, model agnosticism, and auditability. The post highlights the rapid user adoption and its impact on Grab's broader LLM strategy.
  11. Streamlining RiskOps with the SOP agent frameworkThis post details the application of the SOP-driven LLM Agent Framework to automate Account Takeover (ATO) investigations in Risk Operations (RiskOps). It describes the framework's core components: an SOP planner that guides investigations using a tree structure with natural language definitions and @function_name notation for external calls, and a Worker Agent that executes these SOPs. The dynamic execution engine uses a Depth-First Search (DFS) inspired approach for systematic processing. The post highlights the efficiency gains, including a reduction in handling time from 22 to 3 minutes per ticket, 87% automation of ATO cases, and a zero-error rate. It also mentions Slack integration for reporting.
  12. Introducing the SOP-driven LLM agent frameworksIntroduced an LLM agent framework that leverages Standard Operating Procedures (SOPs) represented as a tree structure for guided AI execution. The framework includes an SOP editor for non-technical users, a planner module using Depth-First Search (DFS) with backtracking, an LLM-powered worker agent with context compression and API tool limitations, and a user agent for multilingual interaction. Features like Graph Retrieval-Augmented Generation (GRAG), workflow chaining, and a plugin system (API, Python, SQL) are also included. The framework demonstrated high accuracy (over 99.8%) in use cases like Account Takeover Investigations (ATI) and fraud investigation, significantly reducing investigation time and average handling time.
  13. Grab AI Gateway: Connecting Grabbers to multiple GenAI providersThis post details the evolution of Grab's AI Gateway, a reverse proxy system that unifies access to multiple GenAI providers. Key contributions include the introduction of exploration keys for Grabbers, a unified API interface that translates requests to the OpenAI scheme, dynamic routing for efficient capacity utilization and load balancing, and enhanced auditing and cost attribution mechanisms. The architecture is designed as a lightweight interface with minimal processing time, supporting various API types and in-house models. The post also highlights challenges such as keeping up with rapid innovation and ensuring fair distribution of quota.
  14. Supercharging LLM application development with LLM-KitIntroduced LLM-Kit, a comprehensive framework for building production-ready LLM applications. LLM-Kit provides a pre-configured structure (API server, config management, sample LLM Agent, tests), an integrated tech stack (Poetry, Gunicorn, FastAPI, LangChain, LangSmith, Hashicorp Vault, Amazon EKS, Gitlab CI), built-in observability (Datadog, LangSmith), config/secret management (Python's configparser, Vault), OIDC authentication helpers, API documentation (Swagger, Redoc), Redis & vector database integration, a deployment pipeline, and LangSmith evaluations. It also includes a cookbook with examples like persistent memory agents, Slackbot LLM agents, image analyzers, and full-stack chatbots. The architecture is modular, with automated steps for project generation, code scaffolding, infrastructure provisioning (ECR, EKS, Terraform), and deployment. The scaffolded code includes Agents, Auth, Core, Models, Routes, Storage, Tools, Tracing, and Utils. RAG scaffolding is supported with PGVector integration. Future plans include enhancing web server concurrency/scalability, providing reusable LLM SDKs (evaluation, guardrails), a CLI for updates, and a polling-based agent serving function.