Bayesian Deep Learning and Uncertainty is the framework for quantifying model uncertainty through Bayesian inference — distinguishing epistemic (model) uncertainty from aleatoric (data) uncertainty to enable principled uncertainty estimation for safety-critical applications.

Uncertainty Decomposition:

• Epistemic uncertainty: model uncertainty; reducible with more training data; reflects uncertainty about parameters
• Aleatoric uncertainty: data/measurement uncertainty; irreducible; inherent noise in data generation process
• Total uncertainty: epistemic + aleatoric; total predictive uncertainty crucial for risk-aware decisions
• Heteroscedastic aleatoric: data-dependent noise level; different examples have different noise levels

Monte Carlo Dropout (Gal & Ghahramani):

• Bayesian interpretation: dropout can be interpreted as approximate Bayesian inference via variational inference
• MC sampling: perform multiple forward passes with dropout enabled (stochastic sampling from approximate posterior)
• Uncertainty quantification: variance across stochastic forward passes estimates model uncertainty
• Implementation: trivial modification to existing dropout networks; enable dropout at test time
• Computational cost: requires T forward passes (typically 10-50) per example; tradeoff between accuracy and computation

Deep Ensembles:

• Ensemble uncertainty: train multiple independent models (different initializations, hyperparameters, data subsets)
• Predictive mean: average predictions across ensemble; often better than single model
• Variance estimation: variance of predictions across ensemble estimates model uncertainty
• Aleatoric uncertainty: average predicted variance (if networks output variance) estimates aleatoric uncertainty
• Empirical strong baseline: surprisingly effective; often outperforms more complex Bayesian methods
• Ensemble disadvantage: computational cost proportional to ensemble size; multiple model storage

Laplace Approximation:

• Posterior approximation: approximate posterior as Gaussian around MAP solution; second-order Taylor expansion
• Hessian computation: curvature matrix (Fisher information) captures posterior uncertainty; computationally expensive
• Uncertainty from curvature: high curvature (confident) vs low curvature (uncertain) inferred from Hessian
• Scalability: Hessian computation challenging for large networks; various approximations (diagonal, KFAC) enable scalability

Calibration and Reliability:

• Model calibration: predicted confidence matches true accuracy; miscalibrated models overconfident/underconfident
• Expected calibration error (ECE): average difference between predicted confidence and actual accuracy; measures calibration
• Reliability diagrams: binned predictions showing confidence vs accuracy; visual assessment of calibration
• Temperature scaling: post-hoc calibration; adjust softmax temperature to achieve better calibration without retraining
• Calibration in deep networks: larger networks tend to be miscalibrated (overconfident); calibration essential for safety

Uncertainty Applications:

• Medical diagnosis: uncertainty guiding when to refer to specialist; clinical decision-making support
• Autonomous driving: uncertainty estimates enable collision avoidance; high-risk uncertainty triggers safety protocols
• Out-of-distribution detection: high epistemic uncertainty for OOD inputs; detect dataset shift and anomalies
• Active learning: select uncertain examples for labeling; efficient data annotation strategies

Safety-Critical Deployment:

• Risk-aware decisions: use uncertainty to abstain or request human intervention on high-uncertainty examples
• Confidence calibration: true uncertainty reflects decision quality; essential for safety-critical applications
• Uncertainty feedback: operator informed of model confidence; enables appropriate trust calibration
• Monitoring and drift detection: epistemic uncertainty changes indicate data distribution shift; triggers model retraining

Bayesian deep learning quantifies model and data uncertainty — enabling risk-aware decisions in safety-critical applications where understanding prediction confidence is essential for responsible deployment.