Prediction Intervals

Prediction Intervals are the statistical ranges that quantify the uncertainty in individual predictions — providing upper and lower bounds within which a future observation will fall with a specified probability (e.g., 95%), capturing both the uncertainty in the model's estimated parameters and the inherent randomness of individual outcomes — the essential uncertainty quantification tool that transforms point predictions into actionable ranges for decision-making under uncertainty.

What Are Prediction Intervals?

- Definition: A prediction interval [L, U] for a new observation y_new provides bounds such that P(L ≤ y_new ≤ U) = 1 − α, where α is the significance level (typically 0.05 for 95% intervals). Unlike confidence intervals (which bound parameter estimates), prediction intervals bound individual future observations.
- Two Sources of Uncertainty: (1) Estimation uncertainty — the model's parameters are estimated from finite data and could differ with a different sample, (2) Residual/aleatoric uncertainty — even with perfect parameters, individual observations vary randomly around the predicted value.
- Wider Than Confidence Intervals: Prediction intervals are always wider than confidence intervals because they include both parameter uncertainty AND irreducible observation noise — confidence intervals only capture parameter uncertainty.
- Practical Interpretation: "We are 95% confident that the next observation will fall between L and U" — directly useful for planning, risk assessment, and anomaly detection.

Why Prediction Intervals Matter

- Decision-Making Under Uncertainty: A point prediction of $100K revenue is far less useful than "$85K to $115K with 95% confidence" — intervals enable risk-appropriate decisions.
- Anomaly Detection: Observations falling outside prediction intervals are statistically unusual — prediction intervals provide principled thresholds for anomaly flagging.
- Capacity Planning: Predicting peak load requires upper bounds, not averages — prediction intervals provide the worst-case estimates needed for infrastructure sizing.
- Regulatory Compliance: Medical devices, financial models, and safety-critical systems require uncertainty quantification — point predictions alone are insufficient for regulatory approval.
- Model Calibration Assessment: Checking whether empirical coverage matches nominal probability (e.g., do 95% intervals actually contain 95% of observations?) validates the model's uncertainty estimates.

Prediction Interval Construction Methods

Parametric (Classical Regression):
- For linear regression: PI = ŷ ± t_{α/2} × s_e × √(1 + 1/n + (x − x̄)² / Σ(xᵢ − x̄)²).
- Assumes normally distributed residuals with constant variance.
- Simple and exact for well-specified linear models — breaks down for complex models.

Quantile Regression:
- Train two models: one predicting the α/2 quantile (lower bound) and one predicting the 1−α/2 quantile (upper bound).
- No distributional assumptions — directly estimates conditional quantile functions.
- Works with any regression model (neural networks, gradient boosting, random forests).

Conformal Prediction:
- Distribution-free coverage guarantee: if calibration data is exchangeable with test data, coverage is guaranteed at the nominal level regardless of the underlying distribution.
- Requires a calibration set to compute nonconformity scores.
- Width adapts to local difficulty — wider intervals where the model is less certain.

Ensemble-Based:
- Train multiple models (different initializations, bootstrap samples, or architectures).
- Prediction interval from mean ± k × standard deviation of ensemble predictions.
- Captures model uncertainty through ensemble disagreement; can be combined with residual variance for total uncertainty.

Prediction Interval Comparison

| Method | Distribution-Free | Coverage Guarantee | Width Adaptivity | Complexity |
|--------|-------------------|-------------------|-----------------|------------|
| Parametric | No | Asymptotic | Fixed formula | Low |
| Quantile Regression | Yes | Empirical | Learned | Medium |
| Conformal Prediction | Yes | Finite-sample | Calibration-based | Medium |
| Ensemble | Partially | Empirical | Through disagreement | High |

Calibration Assessment

| Nominal Coverage | Observed Coverage | Interpretation |
|-----------------|------------------|---------------|
| 95% | 95 ± 1% | Well-calibrated ✓ |
| 95% | 88–92% | Under-covering — intervals too narrow |
| 95% | 98–100% | Over-covering — intervals too wide (conservative) |

Prediction Intervals are the language of honest forecasting — transforming point predictions into ranges that acknowledge the irreducible uncertainty in future outcomes, enabling decision-makers to plan for realistic best and worst cases rather than false precision, and providing the calibrated uncertainty quantification that responsible AI deployment demands.