Running Kafka Streams Applications in AWS | Confluent

This post details Zalando's experience running Kafka Streams applications in AWS, highlighting challenges and solutions related to EBS I/O credits, application fine-tuning (timeouts, batch sizes), and memory management (JVM heap vs. off-heap RocksDB usage). It also introduces custom monitoring tools like Remora for consumer lag and emphasizes the importance of comprehensive system monitoring (CPU, memory, disk) for identifying and resolving performance bottlenecks.