Shopify's Path to a Faster Trino Query Execution: Custom Verification, Benchmarking, and Profiling Tooling - Shopify

What this post added

This post details the development of custom verification, benchmarking, and profiling tooling for Trino at Shopify. The tooling is a lightweight Python library designed to minimize the risk of changes to the Trino cluster. The verification component uses PyTest fixtures to run unit tests, including correctness checks (exact row matching) and performance assertions (threshold comparisons on execution time). It supports running queries on single candidate clusters and comparing results against standby control clusters. The benchmarking component uses TPC-DS queries with varying scale factors to evaluate performance under standardized conditions. The post also outlines the problem of managing Trino forks and the need for a structured vetting process, drawing an analogy to Formula 1 race car simulators.