out of distribution

**Out-of-Distribution (OOD) Detection** is the **capability of machine learning models to identify when a test input comes from a different distribution than the training data** — flagging inputs where the model's predictions are unreliable due to distributional shift, enabling AI systems to refuse unreliable predictions rather than confidently generating wrong answers. **What Is OOD Detection?** - **Definition**: Given a model trained on in-distribution data D_in (e.g., X-ray images of lungs), OOD detection identifies inputs from a different distribution D_out (e.g., photos of cats) where the model's learned representations and predictions are not reliable. - **The Silent Failure Problem**: Standard neural networks trained with softmax cross-entropy do not have a native "I don't know" output — when presented with an OOD input, they will output a softmax distribution and often assign high confidence to incorrect classes. - **Famous Example**: A model trained on 10 classes of animals, when shown a random noise image, outputs "Ostrich: 87% confidence" — completely wrong but completely confident. - **Scope**: OOD detection encompasses covariate shift (same labels, different image style), semantic shift (entirely new label categories), and dataset shift (combination of both). **Why OOD Detection Matters** - **Medical AI Deployment**: A chest X-ray classifier trained on adult patients must flag when presented with pediatric patients (different anatomy) rather than confidently misclassifying. - **Autonomous Driving**: A perception system trained on California roads must detect when it encounters conditions outside its training distribution (heavy snow, construction zones with unusual signage) and reduce confidence or request human oversight. - **Industrial Inspection**: A defect detection model deployed on a new product line must recognize when the product has changed beyond its training distribution before falsely passing defective parts. - **Fraud Detection**: A financial fraud model must flag when transaction patterns shift significantly from training data — new fraud patterns are by definition OOD. - **Safety Certification**: Regulatory frameworks for safety-critical AI (FDA SaMD guidelines, automotive SOTIF) increasingly require systems to have OOD detection capabilities with defined confidence bounds. **OOD Detection Methods** **Baseline — Maximum Softmax Probability (MSP)**: - Hendrycks & Gimpel (2017): Simply use max softmax probability as OOD score. - ID inputs typically have higher max softmax probability than OOD inputs. - Simple and surprisingly effective; standard baseline for all subsequent methods. - Limitation: Neural networks are overconfident — OOD inputs often also have high softmax scores. **ODIN (Out-of-DIstribution detector for Neural networks)**: - Liang et al. (2018): Apply temperature scaling + small input perturbations to amplify gap between ID and OOD softmax scores. - Perturbation: x_perturbed = x + ε × sign(∇_x max_c log P(y=c|x)/T). - Significantly outperforms MSP baseline. **Mahalanobis Distance**: - Lee et al. (2018): Fit class-conditional Gaussian distributions in each layer's feature space. - OOD score = minimum Mahalanobis distance from any class mean across all layers. - Requires fitting Gaussians on training data (offline step); strong empirical performance. **Energy-Based OOD**: - Liu et al. (2020): Energy score E(x) = -T × log Σ exp(f_c(x)/T) replaces softmax for OOD detection. - ID inputs have lower energy; OOD inputs have higher energy. - Theoretically grounded in density estimation; training-time energy margin loss further improves detection. **Deep Ensembles for OOD**: - Lakshminarayanan et al. (2017): Ensemble variance provides reliable OOD signal. - Inputs where ensemble members strongly disagree are likely OOD. - High computational cost but strong empirical performance. **Feature Space Density Estimation**: - Train a generative model (normalizing flow, VAE) on training feature representations. - OOD score = negative log-likelihood under the density model. - High-quality but computationally expensive. **OOD Detection Metrics** | Metric | Description | Desired Direction | |--------|-------------|------------------| | AUROC | Area under ROC curve for ID vs OOD | Higher is better (1.0 = perfect) | | AUPR | Area under precision-recall curve | Higher is better | | FPR95 | FPR when TPR = 95% (5% ID rejected) | Lower is better | | Detection accuracy | At optimal threshold | Higher is better | **OOD vs. Related Problems** - **Anomaly Detection**: One-class setting — only ID data available during training; no OOD examples. - **Out-of-Distribution Detection**: Binary classification — ID vs. OOD given examples of both. - **Distribution Shift Detection**: Monitoring for gradual shift in production data over time (data drift). - **Novel Class Discovery**: Identifying OOD inputs that belong to genuinely new semantic categories. OOD detection is **the immune system of deployed AI** — without the ability to recognize inputs that fall outside its training distribution, a model confidently applies learned patterns where they do not apply, generating wrong answers with false certainty. Reliable OOD detection is a prerequisite for safe deployment of AI in any high-stakes domain where inputs cannot be fully controlled.