Tripling streaming pipeline throughput by monitoring and debugging MySQL inefficiencies

This post details the debugging and optimization of MySQL read inefficiencies within Glean's streaming pipeline. It describes how monitoring revealed that large offsets in parallel MySQL queries led to significantly increased read latencies. The core technical contribution is the identification of MySQL's behavior with large offsets and the implementation of a query modification to force late row lookups, resulting in a tripling of pipeline throughput.