How We Optimized Product Syncing for Facebook - Shopify

This post details the optimization of the Facebook channel's product synchronization backend by replacing a serial product sync path with a concurrent one. The solution leverages Sidekiq Pro's batching functionality, where a supervisor job enqueues a batch of product sync jobs that run concurrently. The supervisor holds a lock on the shop's domain to prevent race conditions and re-enqueues itself if further synchronization is needed, increasing worker utility. This change resulted in a decrease of the dataset by 2,000 products per hour and enabled horizontal scalability.