Building Shared State Microservices for Distributed Systems Using Kafka Streams | Confluent

What this post added

This post details how Imperva used Kafka Streams to build shared state microservices for distributed systems. It covers the challenges of traditional state management with single databases, the requirements for new shared state microservices, and how Kafka Streams met these requirements. The post explains the processing topology using Kafka Streams, including source, processor with persistent key-value store, and sink. It also details building a RESTful CRUD API on top of Kafka Streams for state retrieval and updates, distributing processing load across a cluster of microservices for scalability and availability, and implementing task scheduling using the processor context's schedule method. The benefits and challenges of moving from a single microservice to a cluster are also discussed.