uncertainty-based rejection
**Uncertainty-Based Rejection** is a selective prediction strategy that uses estimated prediction uncertainty—rather than raw confidence scores—to decide when a model should abstain from making predictions, routing uncertain inputs to human experts or fallback systems. By leveraging uncertainty estimates from Bayesian methods, ensembles, or MC Dropout, this approach captures model ignorance (epistemic uncertainty) that raw softmax confidence often fails to detect.
**Why Uncertainty-Based Rejection Matters in AI/ML:**
Uncertainty-based rejection provides **more reliable abstention decisions** than confidence thresholding because it directly measures model uncertainty rather than relying on softmax probabilities, which are notoriously overconfident and poorly calibrated for detecting out-of-distribution inputs.
• **Softmax overconfidence problem** — Standard softmax probabilities can assign ≥99% confidence to completely wrong predictions, especially on out-of-distribution inputs; uncertainty-based rejection using ensemble disagreement or Bayesian uncertainty detects these cases that confidence thresholding misses
• **Ensemble disagreement** — When multiple independently trained models disagree on a prediction, the variance across their outputs provides a direct measure of epistemic uncertainty; high disagreement triggers rejection even if individual models appear confident
• **MC Dropout uncertainty** — Running T stochastic forward passes (T=10-50) with dropout enabled at inference produces a distribution of predictions; the variance of this distribution estimates epistemic uncertainty without requiring multiple trained models
• **Predictive entropy** — The entropy of the mean prediction distribution H[E[p(y|x,θ)]] captures both aleatoric and epistemic uncertainty; high predictive entropy triggers rejection as it indicates the model is uncertain about the correct class
• **Mutual information** — The difference between predictive entropy and expected data entropy (mutual information I[y;θ|x,D]) isolates epistemic uncertainty specifically, enabling rejection based on model ignorance rather than inherent class ambiguity
| Method | Uncertainty Source | OOD Detection | Computation Cost |
|--------|-------------------|---------------|-----------------|
| Softmax Confidence | Data only (poor) | Weak | 1× inference |
| Deep Ensemble Variance | Epistemic + Aleatoric | Strong | 5-10× inference |
| MC Dropout Variance | Approx. Epistemic | Good | 10-50× inference |
| Predictive Entropy | Both combined | Moderate | Method-dependent |
| Mutual Information | Pure Epistemic | Strong | Method-dependent |
| Evidential Uncertainty | Distributional | Good | 1× inference |
**Uncertainty-based rejection provides superior abstention decisions by leveraging principled uncertainty estimates that capture model ignorance, detecting unreliable predictions that overconfident softmax scores miss, and enabling robust deployment of AI systems in safety-critical environments where identifying what the model doesn't know is as important as what it does know.**