Law of Large Numbers

The law of large numbers states that as the number of trials of a random process increases, the average of the observed outcomes converges toward the process's true expected value — flip a fair coin 10 times and the proportion of heads might land anywhere from 20% to 80%, but flip it a million times and that proportion will almost certainly sit extremely close to 50%. It is the formal mathematical justification for why larger samples produce more reliable estimates, and it underlies the basic intuition that collecting more data generally improves the accuracy of a statistical estimate, though the precise guarantee the law provides is more specific and more limited than that everyday summary suggests.

The law comes in two formally distinct versions that differ in exactly what kind of convergence they guarantee — the weak law of large numbers, proven by Jacob Bernoulli and published posthumously in 1713 in his work Ars Conjectandi, and the strong law of large numbers, a more rigorous result developed later — and while the technical distinction between the two is a matter of formal probability theory beyond most applied use, the practical takeaway for both is the same: with enough repeated trials, the sample average reliably approaches the true population average, and the probability of a large, persistent deviation from that true average shrinks toward zero as sample size grows. Bernoulli's original proof specifically addressed the weak law in the context of repeated trials with a fixed probability of success, directly connecting the result to the binomial distribution and making it one of the earliest rigorous theorems in the entire history of probability theory.

The gambler's fallacy is the most common and most consequential misapplication of the law of large numbers, mistaking it for a claim it doesn't actually make: the belief that after a run of unusually many heads, tails becomes "due" to balance things out, as if the coin or the universe needs to actively correct the imbalance. The law of large numbers makes no such claim — each individual coin flip remains exactly as independent and exactly as 50/50 as every other flip, regardless of what came before, and the long-run proportion converges toward 50% not because future flips compensate for past ones, but simply because a growing number of future flips dilutes the relative influence of any early, unusual run — the actual count of extra heads from that run stays fixed, it just represents a smaller and smaller fraction of an ever-larger total. Casinos, lotteries, and financial markets have all, at various points, seen real money lost to exactly this misunderstanding.

The law of large numbers specifically describes convergence of an average over many trials, but it says nothing about how fast that convergence happens or how large a deviation to expect at any given, finite sample size — that more precise, quantitative question is answered by a separate but closely related result, the central limit theorem, which describes not just that the sample average converges, but the actual shape and spread of its distribution around the true value as sample size grows, making the two theorems complementary rather than interchangeable: the law of large numbers establishes that convergence happens, the central limit theorem describes how it happens.

In applied data science, the law of large numbers is the theoretical justification behind why A/B tests, surveys, and simulations are run with larger sample sizes when more precision is needed — it's the formal guarantee that collecting more data genuinely moves an estimate closer to the truth on average, which is precisely why "just collect more data" is a statistically sound response to an unreliable estimate, in a way that no comparable guarantee exists for simply re-running a biased sampling method, since the law of large numbers only governs random, unbiased sampling variation, not systematic bias in how a sample was selected in the first place.

Read the law of large numbers as the formal guarantee that more repeated trials move a sample average closer to the true underlying value — not through any kind of self-correcting compensation for past outcomes, which is the exact misunderstanding behind the gambler's fallacy, but simply because each additional independent trial dilutes the relative weight of any earlier unusual run, and its precise complement, the central limit theorem, is what fills in the more practically useful remaining question of how large a deviation from the true value to actually expect at any given, finite sample size.