Catwalk: Serving Machine Learning Models at Scale

This post details the initial implementation of Catwalk, Grab's ML model serving platform. It describes the goals of abstracting infrastructure complexities for data scientists, preventing duplicated effort, and creating a performant, highly available, versioned platform. The core components discussed are TensorFlow Serving for model deployment and Kubernetes for orchestration. TensorFlow Serving is used to load models directly from S3, with automatic versioning and graceful updates. Kubernetes is employed to manage TensorFlow Serving pods, deployments, services, and ingress, enabling autoscaling and high availability. The post outlines the workflow for data scientists to export and upload models to S3, which are then automatically served by Catwalk.