How We’re Solving Data Discovery Challenges at Shopify - Shopify

What this post added

This post introduces Artifact, Shopify's data discovery and management tool, built to address challenges in data curation, governance, and accessibility. It details the tool's user experience, including its search and browse capabilities, data asset details page with lineage visualization powered by a graph database, and its high-level architecture involving metadata ingestion, Elasticsearch indexing, and GraphQL APIs. The post also discusses trade-offs made during development, such as choosing to build over buying an existing solution and opting for a metadata pull strategy over a push strategy.