Evolution of Catwalk: Model serving platform at Grab

This post details the evolution of Grab's ML model serving platform, Catwalk, through four phases. Phase 1 focused on a no-code, managed platform for TensorFlow Serving models, facing scalability and admin bottlenecks. Phase 2 transitioned to a low-code, self-serving model for multiple frameworks (ONNX, PyTorch, TensorFlow) using CI/CD pipelines and introducing Catwalk proxy, transformer, and Amphawa. Phase 3 replaced Helm charts with Kubernetes CRDs for improved deployment control, enabling customisations like smart defaults, blue-green deployments, and advanced scaling. Phase 4 introduced the Catwalk Orchestrator, a high-code framework for building ML applications, supporting bundled deployments and mandatory load testing, leading to a significant increase in deployed orchestrators and models.