Throttling and Rate Limiting

1 post · Aug 2024 – Nov 2024

What this trail covers

This post continues the series on throttlers, exploring the architectural trade-offs between singular and distributed deployments. It delves into the challenges of metrics access, high availability, and scaling for singular throttlers, introducing the concept of agents for metric collection. For distributed throttlers, it discusses various distribution strategies, including per-AZ deployments and functional partitioning. A case study on the Vitess tablet throttler highlights granular control and.

All PlanetScale trails

Nov 2024Anatomy of a Throttler, part 3This post introduces concepts for client identification in throttlers, including the benefits for analysis and operational control (e.g., prioritizing or de-prioritizing specific jobs like ETLs or schema changes). It details the risks of exemption (starvation, system degradation) and proposes prioritization through de-prioritization (e.g., using a dice roll to reject requests for certain clients). The post also discusses the complexities of throttling on different metrics and when exemption might be appropriate (e.g., for incident response or essential system components). It introduces client identity schemes for granular control (e.g., `vreplication:online-ddl`) and emphasizes time-limited rules for exemptions and de-prioritization. Finally, it contrasts cooperative throttling with enforcement-based throttling, using the Vitess transaction throttler as an example.
Oct 2024Anatomy of a Throttler, part 2This post details the architectural considerations for singular vs. distributed throttler deployments. It outlines the challenges of high availability and scaling in singular throttler designs, including the use of agents for metrics access and the introduction of staleness. For distributed throttlers, it explores different distribution models (per-AZ, functional partitioning) and presents a case study of the Vitess tablet throttler, which uses a granular approach with per-vttablet and per-shard throttlers. It also discusses strategies for reducing throttler impact, such as metric hibernation and client-side sleep/retry mechanisms, and the trade-offs involved in metric collection granularity and communication between throttler nodes.
Aug 2024Anatomy of a Throttler, part 1This post, 'Anatomy of a Throttler, part 1,' delves into the design considerations for database system throttlers. It discusses which requests to throttle (asynchronous, batch, massive operations), and what metrics to throttle on, including replication lag, threads_running, and pool usage. It highlights queues as fundamental to understanding these metrics and suggests a combination of metrics for effective throttling.
Advertisement