Making fetch happen: Building a general-purpose query and render scheduler | Datadog
Apr 17, 2023Cormac Flynn

Summary
This post describes the development of a new, generalized query and render scheduler for Datadog applications. The new scheduler simplifies the logic of the previous system by separating query and render scheduling into discrete modules. The query scheduler uses a new algorithm that prioritizes visible widgets and limits concurrent tasks within fixed time windows (2000ms, 10 task limit). The render scheduler leverages the Browser Scheduling API to schedule any sufficiently costly task, improving browser performance and reducing main thread load.
More from the Husky Query Engine trail
- Inside Husky’s query engine: Real-time access to 100 trillion events | Datadog
- Husky: Efficient compaction at Datadog scale | Datadog
- Timeseries indexing at scale | Datadog
- Husky: Exactly-once ingestion and multi-tenancy at scale | Datadog
- Performance improvements in the Datadog Agent metrics pipeline | Datadog