ClickHouse Capacity Estimation Framework
Nov 5, 2020Oxana Kharitonova

Summary
This post details the development of a capacity estimation framework for ClickHouse clusters. It describes the process of identifying relevant metrics (unreplicated data from system.parts), collecting historical data from Thanos using an aiohttp client, forecasting future disk usage with Facebook's Prophet, and visualizing real and predicted data in Grafana. The framework automates capacity planning, making it more transparent and accessible to product engineers.
More from the Data Infrastructure & Analytics trail
- Introducing Cloudflare Basin: an open, serverless data platform, now generally available
- Introducing the Billable Usage API: programmatic cost visibility for Cloudflare
- Announcing the Monetization Gateway: charge for any resource behind Cloudflare via x402
- How we built Cloudflare's data platform and an AI agent on top of it
- Iran's Internet is partially restored, Cloudflare Radar data shows