data drift

Data drift (also called dataset shift or distribution shift) occurs when the statistical properties of the input data that a deployed model receives in production differ from the data it was trained on, potentially degrading model performance over time without any change to the model itself. Data drift is one of the most common causes of model failure in production and a central concern in MLOps — models trained on historical data implicitly assume that future data will follow similar distributions, and when this assumption is violated, predictions become unreliable. Types of data drift include: covariate shift (the distribution of input features changes while the relationship between features and target remains the same — e.g., a customer demographic shifts but the same features still predict the same outcomes), prior probability shift (the distribution of the target variable changes — e.g., fraud rates increase from 1% to 5%), concept drift (the relationship between input features and the target variable changes — e.g., customer preferences evolve, making the same features predict different outcomes), and upstream data changes (alterations in data pipelines, sensor calibration, or data encoding that change the statistical properties of features). Detection methods include: statistical tests (Kolmogorov-Smirnov test, chi-squared test, Population Stability Index comparing training and production feature distributions), distance metrics (Jensen-Shannon divergence, Wasserstein distance between training and production distributions), performance monitoring (tracking prediction accuracy, calibration, and error rates over time — performance degradation suggests drift), and model-based detection (training classifiers to distinguish between training and production data — high accuracy indicates significant drift). Mitigation strategies include: periodic retraining (updating the model on recent data at regular intervals), online learning (continuously updating model parameters with new data), drift-triggered retraining (automatically retraining when drift detection exceeds a threshold), ensemble methods (combining models trained on different time periods), and data preprocessing normalization (reducing sensitivity to distributional changes).