Managing Magic

This post introduces the Data Browser (formerly Data Manager) as a metadata tool to enhance Hex Magic's SQL generation capabilities. It details how the Data Browser improves table retrieval accuracy and reduces string hallucinations by allowing users to provide custom metadata, schema information, dbt documentation, and endorsed statuses. The post explains the underlying mechanism of vector embeddings and provides practical examples of how to use the Data Browser to exclude irrelevant tables, prioritize correct tables, and add nuanced natural language descriptions to guide the AI's SQL generation. It also highlights how custom descriptions are made visible to human users for improved understanding.