Probability Distribution

A probability distribution describes how probability is spread across all the possible values a random variable can take — a complete mathematical description of a random process's behavior, specifying not just what outcomes are possible but how likely each one is relative to the others. A fair six-sided die has a probability distribution that assigns exactly 1/6 probability to each face; a biased die has a different distribution, with probability concentrated unevenly across the same six possible outcomes — same set of possible values, entirely different distribution.

Probability distributions split into discrete and continuous, matching the discrete-versus-continuous split of the random variables they describe, and each type is represented by a different mathematical object: a discrete distribution is described by a probability mass function, listing the exact probability of each individual possible value, while a continuous distribution is described by a probability density function, a curve where the probability of any specific exact value is technically zero and only the area under the curve over a range represents a meaningful probability. A handful of named distributions recur constantly across statistics and data science specifically because they model recurring real-world patterns well: the normal distribution for naturally occurring, symmetric measurement variation (height, measurement error); the binomial distribution for counting successes across a fixed number of independent yes/no trials (defects in a batch); the Poisson distribution for counting events in a fixed interval (customer arrivals per hour); and several others, each matched to a specific generative pattern rather than chosen arbitrarily.

Every probability distribution is fully characterized by its parameters — a small number of values that completely determine its exact shape, which is the practical difference between a distribution family (the general shape, like "normal") and a specific distribution (a particular normal distribution with a defined mean and standard deviation). The normal distribution's two parameters, mean and standard deviation, together determine exactly where it's centered and how spread out it is; two different normal distributions with different parameter values can look dramatically different (one tall and narrow, one short and wide) while still both being, technically and rigorously, "normal distributions" — which is why stating "the data follows a normal distribution" is incomplete without also specifying which normal distribution, meaning which specific mean and standard deviation.

Fitting a probability distribution to real, observed data — estimating which distribution and which specific parameter values best describe the data actually collected — is a core statistical modeling task, typically performed through maximum likelihood estimation, a method that finds the parameter values that would have made the observed data most probable, and it matters because a well-chosen distribution lets an analyst make principled statements about data not yet observed (what's the probability of an even more extreme value occurring), while a poorly matched distribution produces confident-looking but ultimately unreliable extrapolations, since the mathematical machinery keeps running even when the underlying distributional assumption doesn't actually hold.

Every valid probability distribution must satisfy the constraint that its total probability sums (for discrete distributions) or integrates (for continuous distributions) to exactly 1 across all possible outcomes, a direct consequence of Kolmogorov's probability axioms — this constraint is what ties any specific, named distribution back to the formal mathematical definition of probability itself, ensuring that no matter how the probability is shaped or spread across possible values, something is guaranteed to happen with total certainty.

Read a probability distribution as the complete mathematical fingerprint of a random process, far more informative than any single summary statistic alone: its discrete-versus-continuous classification determines whether a probability mass function or density function describes it, its specific parameters (not just its general family) are what fully pin down its shape, and correctly matching the right distribution to the real-world process generating the data — rather than defaulting to whichever distribution is most familiar — is one of the most consequential modeling decisions in applied statistics, because every formula and test built on top of a distributional assumption inherits whatever error that initial mismatch introduces.