Reduced Message Throughput | Diagnose and Debug Kafka Issues Series

This post provides a detailed guide to diagnosing reduced message throughput in Apache Kafka. It focuses on producer-side metrics such as batch-size-avg, records-per-request-avg, record-size-avg, record-queue-time-avg, request-latency-avg, and requests-in-flight. It also explains how to tune producer configurations like batch.size, linger.ms, and buffer.memory to optimize batching. Additionally, it suggests investigating topic partitions, request rates, and the number of KafkaProducer instances as potential root causes for throughput issues.