How to Limit Memory Usage in ksqlDB

This post details how ksqlDB manages memory usage, focusing on bounding RocksDB's off-heap memory through shared caches and write buffer managers. It investigates a real-world scenario of unbounded memory growth, identifies the root cause in glibc's malloc behavior with terminated queries not releasing memory effectively, and proposes solutions. The post highlights the effectiveness of alternative allocators like jemalloc and TCMalloc, and explains the internal mechanics of glibc's arenas and heaps to understand memory retention. It also details how Confluent Cloud addresses this by configuring JVM heap and RocksDB limits, and by implementing a KsqlBoundedMemoryRocksDBConfigSetter to manage shared caches and write buffers.