Real-Time AI: Live Recommendations Using Confluent and Rockset

This post details how Confluent and Rockset enable real-time AI applications by providing a robust architecture for contextualizing AI models with fresh data. It explains two patterns for contextualizing models: extending models with real-time data and feeding models with real-time data. The post highlights the challenges of traditional batch-oriented data pipelines and introduces Confluent Cloud as a data streaming platform and Rockset as a real-time analytics database with vector search capabilities. It uses the example of Whatnot, an e-commerce innovator, to illustrate how this pattern is used for real-time recommendations based on user activity and livestream data. Key technical aspects include low data latency, high query latency, efficient upserts, metadata filtering alongside vector search, and compute-compute separation for scaling.