Kafka Broker and Streams Enhancements
Introducing Confluent’s Parallel Consumer Message Processing Client

Introducing Confluent’s Parallel Consumer Message Processing Client

12/15/2020 · Antony Stubbs

What this post added

Introduces the Confluent Parallel Consumer, a new client library that provides key-level and unordered parallelism for Kafka message consumption, surpassing the limitations of partition-level parallelism. It details how this library enables higher throughput by allowing more concurrent processing than available partitions and by efficiently handling external service calls (e.g., HTTP requests to legacy services) through non-blocking I/O and optional Vert.x integration. The post also highlights features for client-side work queues and message-level acknowledgment, improving performance for task queue use cases.

Read the original post ↗