Markov Chain Monte Carlo (MCMC) is a family of algorithms that generate samples from a target probability distribution (typically a Bayesian posterior p(θ|D)) by constructing a Markov chain whose stationary distribution equals the target distribution. MCMC enables Bayesian inference for models where direct sampling or analytical computation of the posterior is intractable, requiring only the ability to evaluate the unnormalized posterior p(D|θ)·p(θ) up to a proportionality constant.

Why MCMC Matters in AI/ML: MCMC provides asymptotically exact Bayesian inference for arbitrary probabilistic models, making it the gold standard for posterior estimation when computational budget permits, and the reference against which all approximate inference methods are evaluated.

• Metropolis-Hastings algorithm — The foundational MCMC method: propose θ from a proposal distribution q(θ|θ_t), accept with probability min(1, [p(θ|D)·q(θ_t|θ)]/[p(θ_t|D)·q(θ*|θ_t)]); the chain converges to the target distribution regardless of initialization given sufficient iterations • Gibbs sampling — A special case of MH where each parameter is sampled from its full conditional distribution p(θ_i|θ_{-i}, D), cycling through all parameters; especially efficient when conditionals have known distributional forms • Convergence diagnostics — Multiple chains from different initializations should produce consistent estimates; R-hat (potential scale reduction factor) < 1.01, effective sample size (ESS), and trace plots assess whether the chain has converged and mixed adequately • Burn-in and thinning — Initial samples (burn-in) are discarded as the chain has not yet converged to the stationary distribution; thinning (keeping every k-th sample) reduces autocorrelation but is generally less effective than running longer chains • Stochastic gradient MCMC — For large datasets, SGLD and SGHMC use mini-batch gradient estimates with injected noise to perform MCMC without full-dataset evaluations, enabling MCMC for neural network-scale models

MCMC is the foundational methodology for Bayesian computation, providing asymptotically exact posterior samples for arbitrary probabilistic models through the elegant construction of convergent Markov chains, serving as both the practical workhorse for Bayesian statistics and the theoretical benchmark against which all approximate inference methods are measured.