How Shopify improved consumer search intent with real-time ML - Shopify

What this post added

This post details the implementation of real-time ML inference streaming pipelines for generating embeddings for Shopify Storefront Search. It covers the architecture using Google Cloud Dataflow, the preprocessing steps for images, and the challenges encountered and solutions implemented, including managing data in memory by adjusting Dataflow worker threads and the trade-offs in managing the ML model in memory. It also discusses the importance of batching for GPU utilization and the implementation of batching via Apache Beam's ModelHandler.