TechPrufer
Get started
Shopify
Shopify logoShopify · Feature trail

Product Categorization System

6 posts · Apr 2020 – Jun 2026

What this trail covers

Shopify has developed and enhanced its product categorization system by introducing an AI multi-agent system to continuously evolve the taxonomy. This system goes beyond static classification by actively improving taxonomy labels themselves, keeping the system agile and future-proof. It integrates actual merchant product data for real-world grounding, employs specialized agents for structural consistency and product-driven insights, synthesizes these insights intelligently, and detects complex e. This post details the evolution to a Vision Language Model-based system that leverages a comprehensive product taxonomy for granular understanding, attribute extraction, and description generation. The system employs FP8 quantization, in-flight batching, and KV cache optimization for inference efficiency. Training data quality is ensured through a multi-LLM annotation system with human validation. Future directions include migrating to a DAG structure for the taxonomy and enhancing metadata extraction.

All Shopify trails
  1. Clustering billions of products for agentic commerce with Catalog API (2026) - ShopifyThis post details the development of a product clustering system for Shopify Catalog, which aims to create a unified intelligence layer for product data. It addresses the challenge of standardizing product listings across millions of stores for AI shopping agents. The system uses a two-stage LLM pipeline for intra-store clustering, starting with a singleton detector to identify products that don't require clustering. For shops that do require clustering, it employs Approximate Nearest Neighbor (ANN) retrieval with HNSW and a modified average-linkage approach (UPGMA) to create semantically related product neighborhoods (chunks) before LLM processing. The core principle for LLM prompting is the 'core value proposition' framework, which determines if an attribute is a variant or defines product identity. Precision is prioritized over recall, with quality measured using curated ground-truth datasets and LLM-based judges.
  2. How AI Agents are Evolving Shopify's Product Taxonomy at Scale (2025) - ShopifyThis post details the development of an AI multi-agent system for continuous product taxonomy evolution. It introduces specialized agents for structural consistency and product-driven insights, intelligent synthesis of these insights, and a novel equivalence detection mechanism for categories filtered by attribute values. The system grounds taxonomy decisions in real merchant product data and includes automated quality assurance with AI judges.
  3. Leveraging multimodal LLMs for Shopify’s global catalogue: Recap of expo talk at ICLR 2025 - ShopifyThis post details the engineering behind Shopify's Global Catalogue, which uses multimodal LLMs to organize and augment product data. It covers data curation, model fine-tuning, and the infrastructure for daily inferences. Key technical contributions include: a multi-task, multi-entity approach for product understanding using fine-tuned vision LLMs (LlaVA 1.5 7B, LLaMA 3.2 11B, Qwen2VL 7B); selective field extraction during fine-tuning to improve generalization and reduce latency/GPU usage; an automated annotation pipeline using LLM agents and human expertise for data generation; a product matching system using bipartite graphs and discriminator models for clustering related listings; and a reconciliation layer to construct canonical product records. The post also discusses the challenges of fragmented product data and the impact of these pipelines on search, recommendations, and conversational commerce.
  4. Augmented commerce: Machine learning at Shopify (2025) - ShopifyThis post details Shopify's use of finetuned Qwen multimodal models for classifying and enriching product metadata, handling hundreds of millions of inferences daily. It also mentions the use of LLaMa models and general models for Sidekick, Nomic embeddings for product representations, tabular transformer models for GMV forecasting, and experimentation with HSTU architecture for merchant/customer behavior, as well as exploring small language models for query rewriting.
  5. Evolution of Product Classification at Shopify: From Categories to Comprehensive Product Understanding (2025) - ShopifyThis post details the evolution of Shopify's product classification system from basic ML models to a multi-modal approach and finally to a Vision Language Model (VLMs) based system integrated with the Shopify Product Taxonomy. Key technical contributions include: 1. Use of VLMs (LlaVA 1.5 7B, LLaMA 3.2 11B, Qwen2VL 7B) for true multi-modal understanding, zero-shot learning, and natural language reasoning. 2. Inference optimizations: FP8 quantization for reduced GPU memory and efficient batching, in-flight batching using Nvidia Dynamo for dynamic request grouping and adaptive processing, and KV cache optimization for LLM inference speed. 3. A near real-time processing pipeline orchestrated by Dataflow, making two calls to the VLM service (category and attribute prediction), with transaction-like handling and retries. 4. A multi-stage annotation system for training data quality, involving multiple LLMs and human validation. 5. Impact metrics showing an 85% acceptance rate of predicted categories, doubled hierarchical precision and recall, and daily processing of over 30 million predictions. Future plans include migrating to a DAG taxonomy and enhancing metadata extraction.
  7. Using Rich Image and Text Data to Categorize Products at Scale - ShopifyThis post details the modernization of Shopify's product categorization model, which now incorporates both rich image and text data. The new model uses transfer learning with pre-trained Multi-Lingual BERT for text and MobileNet-V2 for images, converting them into embeddings. The architecture is framed as a multi-task, multi-class classification problem, with each level of the Google Product Taxonomy (GPT) treated as a separate classification task. The output of each layer is fed back into the next to improve predictions. The model is trained using distributed TensorFlow on Google Cloud Platform with data parallelization to handle the massive dataset and model complexity. During inference, additional logic is applied to ensure hierarchical consistency, selecting the highest confidence category at Level 1 and then choosing the highest confidence descendant at subsequent levels. This approach increased leaf precision by 8% and doubled coverage.
  8. Categorizing Products at Scale - ShopifyThis post details the implementation of a product categorization system using Kesler's Construction to convert n one-vs-all classifiers into a single binary classifier. It leverages Logistic Regression and a greedy approach for inference on a hierarchical taxonomy. Text featurization uses HashingTF with PySpark, focusing on product title, description, collection, tags, vendor, and merchant-provided type. The system addresses scale and structure challenges of the Google Product Taxonomy (GPT) and is used by over 20 teams for features like marketing recommendations and market segmentation.