Building a Hyper Self-Service, Distributed Tracing and Feedback System for Rule & Machine Learning (ML) Predictions

This post introduces Archivist, a new tracing, statistics, and feedback system for ML/rule-based prediction events. Archivist addresses limitations of previous logging methods (plain text logs, lack of fine-granularity access control, separate Elasticsearch clusters per team) by providing a centralized, performant, and flexible solution. Key improvements include user-defined entities and custom fields, a shared event processor that writes to a central Elasticsearch cluster and S3 data lake, and a user portal for querying, prediction stats, and feedback. It reduces development time for new business scenarios and improves data accessibility for different roles within Grab.