Using Rich Image and Text Data to Categorize Products at Scale - Shopify

What this post added

This post details the evolution of Shopify's product categorization system by introducing a new model that incorporates rich image and text data. It describes the use of multi-lingual BERT for text embeddings and MobileNet-V2 for image embeddings. The model architecture is framed as a multi-task, multi-class classification problem, where each level of the Google Product Taxonomy (GPT) is a separate classification task, with outputs feeding into subsequent levels. Training is optimized using data parallelization on Google Cloud Platform. Inference logic is enhanced to enforce hierarchical consistency, ensuring accurate and reliable predictions for consumers. The new model achieved an 8% increase in leaf precision and doubled coverage.