
5/8/2025
What this post added
This post details the evolution of Shopify's product understanding system, moving from early basic classification with logistic regression and TF-IDF to a multi-modal approach combining image and text data. The current generation is built on Shopify's Standard Product Taxonomy and Vision Language Models (VLMs). It introduces the technical deep dive into inference optimization, including FP8 quantization, in-flight batching, and KV cache optimization. The pipeline architecture is described, highlighting a two-stage prediction process for categories and attributes using a Dataflow pipeline and a Kubernetes cluster with NVIDIA GPUs. The post also elaborates on the robust training data system, which uses a multi-LLM annotation system with arbitration and human validation. The impact is quantified with an 85% acceptance rate of predicted categories and doubled hierarchical precision and recall. Future directions include incorporating new VLM architectures, expanding attribute prediction, and migrating to a DAG taxonomy.