Data-Centric Machine Learning: Building Shopify Inbox’s Message Classification Model - Shopify

This post details the evolution of Shopify Inbox's message classification model by applying a data-centric approach. It describes the transition from an initial prototype using Universal Sentence Encoder and clustering techniques to a more robust system utilizing DistilBERT. Key contributions include the development of a new taxonomy with over 40 topics grouped into five categories (Products, Pre-Purchase, Post-Purchase, Store, Miscellaneous), a systematic annotation process with inter-annotator agreement measurement, and the fine-tuning of the DistilBERT model for improved accuracy and coverage. The post highlights the importance of high-quality training data and the benefits of a data-centric ML process.