Independent Events

Two events are independent when the occurrence of one does not change the probability of the other, which in formula form means the probability of both occurring equals the product of their individual probabilities: P(A and B) = P(A) × P(B). Equivalently, the conditional probability P(A given B) equals P(A). Two successive tosses of a fair coin are independent, so the chance of two heads is 1/2 × 1/2 = 1/4, and knowing the first toss landed heads gives no information about the second. Independence is a statement about information and probability, not about physical separation or causal links, and it is the assumption that makes complex probability calculations multiply neatly.

The multiplication rule for independent events is what makes long chains of random trials tractable. The probability that five independent components all work, each with probability 0.99, is 0.99 to the fifth power, about 0.951; the probability of ten consecutive heads is 1/1,024. Independence also underwrites the binomial distribution, which counts successes in a fixed number of independent trials, and the i.i.d. (independent and identically distributed) assumption that nearly all classical statistics uses: standard errors, confidence intervals and the central limit theorem all rely on observations that do not inform one another. When the assumption holds, the algebra collapses; when it fails, the calculation changes substantially.

Independence must never be confused with mutual exclusivity, and the two properties pull in opposite directions. Mutually exclusive events cannot both occur, so P(A and B) = 0; for events with positive probability that can never equal P(A) × P(B), so they are dependent, because one event's occurrence guarantees the other's absence. Independent events with positive probabilities, by contrast, must overlap: the die-and-coin example shares one outcome in twelve. A quick test distinguishes them: if P(A and B) is zero, the events are exclusive (and dependent); if it equals the product, they are independent; otherwise they are dependent but overlapping. Beginners often assume that "unrelated" events are exclusive, or that exclusive events are "independent because they don't affect each other," and both inferences are mistaken.

The gambler's fallacy and the misuse of independence in the Sally Clark case show the stakes of getting the concept right. The gambler's fallacy is the belief that after a run of red on a roulette wheel, black is "due"; because spins are independent, the wheel has no memory, and the commonly cited illustration is the Monte Carlo casino night of August 1913, when roulette results landed on black a very long run of times. The opposite error, assuming independence where it fails, had graver consequences in the 1999 conviction of Sally Clark in England for the deaths of her two infants. A pediatrician's testimony that the chance of two cot deaths in one family was about 1 in 73 million came from squaring a single-death figure, ignoring shared genetic and environmental factors that make the events dependent. The Royal Statistical Society publicly criticized the calculation in 2001, and her conviction was overturned on appeal in 2003.

Independence comes in several flavors that matter in practice. A set of events is pairwise independent if every pair satisfies the product rule, but mutual independence demands the product rule for every sub-collection, a stronger condition; Sergei Bernstein gave a well-known example showing three events that are pairwise but not mutually independent. Events can also be conditionally independent given a third, as when two symptoms are unrelated once the disease is known, which is the key assumption of the naive Bayes classifier and the basis of graphical models and Bayesian networks. Sampling without replacement produces dependence (drawing a second ace from a deck depends on the first draw), though for small samples from a very large population the dependence is negligible, which is why surveys of 1,000 people from a nation of millions are treated as independent draws.

Real-world data are rarely independent, so verifying the assumption is part of the analysis. Time series, users in a social network, repeated measurements on the same patient, and students in the same classroom are all clustered or autocorrelated, and treating them as independent understates uncertainty and inflates apparent significance. Remedies include mixed-effects models, clustered standard errors and block bootstrapping. In machine learning, leakage between training and test sets is a violation of independence that yields overly optimistic performance. Common checks include examining autocorrelation, running tests of association such as chi-square for categorical variables, and thinking carefully about how the data were collected.

Read independent events as events that carry no information about each other: the product rule is the formal test, it is not the same as being exclusive (exclusive events are the extreme case of dependence), and the quiet assumption of independence is one of the most powerful and most frequently violated simplifications in all of statistics.