Posterior Probability

A posterior probability (or posterior distribution) is the probability of an unknown quantity after the observed data have been taken into account: the prior belief reweighted by how well each possible value explains what was seen, then renormalized so that it sums or integrates to one. It is the output of the update described under bayesian-inference and the counterpart of the prior covered under prior-probability; this entry is about what the posterior means and how to read, summarize and use it. Formally it is the conditional probability of the unknown given the data, proportional to prior × likelihood, with the normalizing constant supplied by bayes-theorem.

The posterior is a whole distribution, and reducing it to one number throws away the very thing that makes it valuable, which is the quantified uncertainty. Common summaries each answer a different question: the posterior mean minimizes expected squared error, the posterior median minimizes expected absolute error, and the mode (the maximum a posteriori or MAP estimate) is the single most probable value and the one closest to the output of a regularized optimizer. In the standard worked example of ten coin flips with seven heads and a uniform Beta(1, 1) prior, the posterior is Beta(8, 4), with mean 8/12 ≈ 0.67, mode 7/10 = 0.70 and a long left tail; the probability that the coin favors heads (p above 0.5) is about 0.89. That last sentence is a direct probability statement about a parameter, which is precisely what the posterior is for and what a frequentist p-value cannot supply.

A credible interval is a direct probability statement, and that is the difference between it and a frequentist confidence interval, which people constantly misread as one. A 95% credible interval says that, given the model and prior, the parameter lies in the interval with probability 0.95. A 95% confidence interval says something about the procedure: if the experiment were repeated many times, 95% of intervals built this way would cover the true value, but any single interval either contains it or does not. Credible intervals come in two common forms, the equal-tailed interval, which cuts 2.5% off each end, and the highest posterior density interval (HDI), the shortest interval holding the stated mass, which is better for skewed posteriors. Researchers who want to claim a practically meaningful effect often compare the posterior to a region of practical equivalence (ROPE) around zero, and report the proportion of posterior mass beyond it, instead of a bare tail probability.

Predictions should average over the posterior rather than plug in a single estimate, which is the role of the posterior predictive distribution. The probability of a new observation is the integral of the likelihood of that observation over the posterior of the parameters, so it automatically fattens its tails to reflect parameter uncertainty; a plug-in prediction from a maximum-likelihood-estimation fit is typically overconfident, especially with little data. The same distribution drives model checking: simulate replicated datasets from the posterior predictive and compare them with the real data, and systematic discrepancies show where the model fails. When the posterior has no closed form, it is represented by thousands of draws from markov-chain-monte-carlo, and every summary above becomes a simple calculation on those samples: a mean of the draws, a quantile, a count of how many exceed a threshold. In a Gaussian setting the posterior may itself be normal, and then its mean and variance are all that is needed.

Posterior beliefs depend on the model and the prior, and they are not corrections of reality that the data guarantee. A posterior can be extremely narrow around a wrong value if the likelihood is misspecified (assuming independence between observations that are in fact correlated is a typical culprit), and it can be dominated by the prior when data are sparse. Another frequent error is the "prosecutor's fallacy" version in Bayesian clothing: confusing the probability of the data given a hypothesis with the probability of the hypothesis given the data. The posterior also has no automatic meaning for a long-run frequency; it expresses plausibility, and good calibration, meaning that 90% intervals actually cover about 90% of true values, has to be checked by simulation or held-out data. Finally, a posterior from one analysis becomes the prior for the next, which is the sequential rule described in the inference entry, but only if the data in the two steps are genuinely conditionally independent given the parameters.

Read a posterior as the complete, updated map of what remains plausible once the data are in, best used through intervals, event probabilities and predictions rather than collapsed to a single number, and only ever as reliable as the model that produced it.