Grab AI Gateway: Connecting Grabbers to multiple GenAI providers

This post introduces the Grab AI Gateway, a system designed to provide unified and streamlined access to multiple Generative AI (GenAI) providers (OpenAI, Azure, AWS Bedrock, Google VertexAI, etc.) for Grabbers. It details the architectural design as a set of reverse proxies, the IAM implementation using API keys with request path-based authorization, and the user journey including onboarding, exploration keys, a unified API interface (based on OpenAI's scheme), dynamic routing for usage efficiency, auditing of calls, cost attribution, and rate limiting. It also highlights integration with Grab's ML platform (Chimera notebooks, Catwalk). Challenges include keeping up with rapid innovations, fair distribution of quota between batch and online use cases, and maintaining reverse proxies.