Cloudflare

ClickHouse Capacity Estimation Framework

Nov 5, 2020Oxana Kharitonova
ClickHouse Capacity Estimation Framework
SummaryAI, from the source post

This post details the development of a capacity estimation framework for ClickHouse clusters. It describes the process of identifying relevant metrics (unreplicated data from system.parts), collecting historical data from Thanos using an aiohttp client, forecasting future disk usage with Facebook's Prophet, and visualizing real and predicted data in Grafana. The framework automates capacity planning, making it more transparent and accessible to product engineers.

More from the Data Infrastructure & Analytics trail