Data Mesh at Grab (Part II): The foundational tools behind certification

This post details the foundational tools for data mesh certification at Grab: Hubble (metadata management and certification engine) and Genchi (data quality observability). Hubble, built on DataHub, provides search, ownership, data contracts, lineage, and computational certification status. Its event-driven architecture ensures metadata stays fresh. Genchi allows defining data quality tests (freshness, volume, schema, semantic validity) and integrates with pipeline completion events via Lighthouse's Sync with Pipeline (SWP) for event-driven test orchestration, moving away from cron-based schedules to ensure tests align with actual data production.