The Magic of Merlin: Shopify's New Machine Learning Platform - Shopify

What this post added

This post introduces Merlin, Shopify's new machine learning platform, built on an open-source stack including Ray. It details Merlin's architecture, focusing on Merlin Workspaces (Ray clusters on Kubernetes) for training and batch inference, and Merlin Projects for defining dependencies. The post explains how users can prototype in Jupyter Notebooks connected to these workspaces via the Ray Client API and productionize workflows using tools like Airflow. It highlights Ray's role in enabling distributed ML workflows in Python and its integrations like Ray Train, Ray Tune, and the Ray Kubernetes Operator.