Trident - Real-time Event Processing at Scale

This post introduces Trident, Grab's in-house real-time IFTTT engine for automating business workflows and campaigns. It details the system's architecture, which consumes events from Kafka, ensures exactly-once processing using Redis and MySQL, and scales to handle billions of events daily. Key technical challenges and solutions discussed include load distribution for Kafka partitions and server auto-scaling, and performance optimizations for rule evaluation through event prefiltering and weighted rule evaluation with lazy loading. The post highlights the system's ability to process over 2,000 events per second and deliver massive volumes of rewards and communications.