TechPrufer
Get started
Hex
Hex logoHex · Feature trail

Data Drift Detection and Mitigation

4 posts · Jan 2026 – May 2026

What this trail covers

Hex has evolved its data trustworthiness capabilities by introducing comprehensive explanations and strategies for data drift. This includes detailing the causes of data drift (silent schema changes, SaaS platform modifications, third-party format changes, upstream pipeline rewrites, semantic drift), its impact on AI analytics (corrupting answers, passing quality checks while degrading), and methods for detection at the data-team level (statistical tests like KS, Wasserstein distance, PSI, chi-s. This post details data quality monitoring techniques, covering both structural and semantic dimensions, and how AI can be leveraged for automated anomaly detection and validation against learned patterns. It emphasizes integrating these checks into the analytics workflow for continuous assessment of data fitness.

All Hex trails
  1. Data Drift Explained: Causes, Detection & FixesThis post details the causes, detection methods, and response strategies for data drift. It elaborates on silent schema changes, SaaS platform modifications, third-party format changes, upstream pipeline rewrites, and semantic drift as causes. For detection, it introduces statistical tests like Kolmogorov-Smirnov (KS), Wasserstein distance, Population Stability Index (PSI), and chi-squared tests, alongside schema change detection and integration with dbt, Airflow, and Dagster. The post also outlines a governed response, emphasizing immediate flagging of affected tables and the importance of feedback loops.
  2. What Is Model Drift? Causes, Detection, and How to Fix ItThis post details model drift, distinguishing between data drift, concept drift, prediction drift, training-serving skew, and upstream drift. It emphasizes that most drift issues stem from upstream data quality or pipeline changes rather than the model itself. Practical detection methods are described, including storing training baselines, using Population Stability Index (PSI) for numerical and categorical features, and setting alert thresholds based on business impact and use-case risk. It also highlights limitations of input monitoring and suggests user query patterns as an early signal. The post provides a table mapping symptoms to root causes and first responses, stressing the importance of fixing data issues before retraining and discussing when retraining is appropriate.
  3. Machine Learning Failure Modes: Why Production Models BreakThis post details the diagnosis and mitigation of underfitting, including identifying it through high training and validation errors, understanding its relation to the bias-variance tradeoff, and outlining a systematic approach to fixing it by increasing model capacity, reducing regularization, engineering better features, improving data quality, and systematically tuning hyperparameters. It also highlights the importance of data leakage and shortcut learning as common failure modes in ML systems, emphasizing that production ML failures cluster into four categories: data issues, shortcut learning, system complexity, and organizational failures. The post advocates for proactive monitoring of input data distributions using statistical tests like Kolmogorov-Smirnov, Population Stability Index, and Jensen-Shannon Divergence to catch model degradation early, and stresses the importance of temporal validation and auditing features to prevent data leakage. It also introduces worst-group-accuracy as a metric to test for shortcut learning and discusses advanced techniques like counterfactual data augmentation and invariant risk minimization.
  4. Data quality monitoring: what to track & howThis post details data quality monitoring techniques, covering both structural (accuracy, completeness, consistency, timeliness, validity, uniqueness) and semantic (semantic clarity, source trustworthiness, contextual completeness) dimensions. It outlines traditional techniques like data profiling, anomaly detection, data lineage tracking, schema monitoring, and CI/CD integration. It also introduces AI-powered monitoring, emphasizing automated baseline learning and its application to both traditional and semantic dimensions. The post highlights the importance of semantic layers for AI-driven semantic appropriateness and advocates for integrating monitoring into the analytics workflow.