Improving Meta’s SLO workflows with data annotations

This post introduces enhancements to Meta's SLICK SLO management platform by integrating a collaborative data annotation framework. It details the challenges of identifying failure patterns from unstructured SLO violation data and proposes a solution involving structured annotations. The post describes the development of a flexible annotation schema, various methods for creating schematized annotations (CLI, UI, Workplace bot, and automated integration with Dr. Patternson), and an insights UI for visualizing aggregated annotation data. The key technical contributions include the design of the `annotation_config` field for defining hierarchical matchers, the implementation of interactive annotation creation tools, and the development of an insights dashboard for analyzing root causes and metadata distributions.
More from the Site Reliability and Outage Management trail
- Lights Out, Systems On: Validating Instant Power Loss Readiness
- DrP: Meta’s Root Cause Analysis Platform at Scale
- Zoomer: Powering AI Performance at Meta’s Scale Through Intelligent Debugging and Optimization
- Diff Risk Score: AI-driven risk-aware software development
- How Meta keeps its AI hardware reliable