Shopify's Path to a Faster Trino Query Execution: Infrastructure - Shopify
Jul 9, 2021
Summary
This post details the optimization of Shopify's Trino query execution infrastructure by addressing issues related to JVM settings, CPU and memory allocation, and workload-specific cluster configurations. This includes implementing workload-specific clusters for ad-hoc, scheduled, and experimental queries, reducing coordinator lock contention by setting a hard concurrency limit, and tuning JVM settings and resource allocation to improve cluster throughput and query latency.
More from the Data Store Selection and Usage trail
- We replaced Redis with MySQL for inventory reservations—and it scaled (2026) - Shopify
- Mitigating Deadlocks in high concurrency environments - Shopify
- Safely Adding NOT NULL Columns to Your Database Tables - Shopify
- Double Entry Transition Tables: How We Track State Changes At Shopify - Shopify
- Shopify’s Unique Data Science Hierarchy Of Needs - Shopify