domain shift
**Domain shift** (also called distribution shift) occurs when the **statistical distribution of test/deployment data differs** from the distribution of training data. It is one of the most common and impactful causes of model performance degradation in real-world AI deployments.
**Types of Domain Shift**
- **Covariate Shift**: The input distribution P(X) changes, but the relationship P(Y|X) stays the same. Example: A model trained on professional photos struggles with smartphone photos — the subjects are the same but the image quality differs.
- **Label Shift (Prior Probability Shift)**: The output distribution P(Y) changes. Example: A disease diagnostic model trained when prevalence was 5% deployed when prevalence rises to 20%.
- **Concept Drift**: The relationship P(Y|X) itself changes — the same inputs should now produce different outputs. Example: Fraud patterns evolve over time.
- **Dataset Shift**: A general term encompassing any distributional difference between training and deployment data.
**Why Domain Shift Happens**
- **Temporal Changes**: The world changes over time — user behavior, language, trends, and data distributions evolve.
- **Geographic Differences**: A model trained in one region encounters different demographics, languages, or cultural contexts in another.
- **Platform Changes**: Data collected from different devices, sensors, or software versions has different characteristics.
- **Selection Bias**: Training data was collected differently than deployment data (e.g., hospital data vs. field data).
**Detecting Domain Shift**
- **Performance Monitoring**: Track model accuracy on labeled production data — degradation suggests shift.
- **Distribution Comparison**: Compare input feature distributions between training and production data using KL divergence, MMD, or statistical tests.
- **Drift Detection Algorithms**: DDM, ADWIN, and other algorithms detect distributional changes in data streams.
**Mitigating Domain Shift**
- **Domain Adaptation**: Explicitly adapt the model to the new domain using techniques like fine-tuning or domain-adversarial training.
- **Domain Generalization**: Train the model to be robust across domains from the start.
- **Continuous Learning**: Periodically retrain or update the model on recent data.
- **Data Augmentation**: Expose the model to diverse conditions during training.
Domain shift is the **primary reason** ML models degrade after deployment — monitoring for and adapting to distribution shifts is essential for maintaining production model quality.