Bernoulli Distribution

The Bernoulli distribution is the probability distribution of a single trial with exactly two possible outcomes, conventionally coded 1 for success and 0 for failure, where success occurs with probability p and failure with probability 1 − p. It is the smallest nontrivial distribution there is, described by one number, and it models any yes/no event: a coin landing heads, a customer clicking an ad, a patient responding to a treatment, a manufactured part passing inspection. It is named for the Swiss mathematician Jacob Bernoulli, whose Ars Conjectandi, published posthumously in 1713, proved an early form of the law of large numbers for exactly this kind of repeated two-outcome trial.

Because the outcome is coded 0 or 1, the distribution's summary statistics collapse into neat formulas: the mean is p and the variance is p × (1 − p). The expected value of a variable that is 1 with probability p and 0 otherwise is simply p, so the average of many 0/1 results is a proportion, which is why a sample proportion is nothing more than a sample mean of Bernoulli data. The variance p(1 − p) is largest, at 0.25 (a standard deviation of 0.5), when p = 0.5 and shrinks toward zero as p approaches 0 or 1, matching intuition that a nearly certain event is nearly constant. Notice that this is unlike the normal-distribution, where mean and spread are independent dials: for a Bernoulli variable, the spread is fully determined by the mean.

Bernoulli trials are the atoms from which several better-known distributions are built. Add up n independent Bernoulli(p) trials and the total is a binomial-distribution with parameters n and p, so a Bernoulli is simply the binomial with n = 1. Count how many independent trials it takes to get the first success and the result is the geometric distribution. When n is large and p is small, the number of successes is well approximated by the poisson-distribution, and when n is large the standardized count approaches normality, one of the earliest forms of the central-limit-theorem (de Moivre and Laplace). In every case the building blocks must satisfy two conditions, independence of trials and a constant p, and when either fails the derived distributions fail with it.

In data science and machine learning, the Bernoulli distribution is the standard model for any binary label. Logistic regression assumes each outcome is a Bernoulli variable whose p is a logistic function of the predictors, and fitting it by maximum likelihood means maximizing the Bernoulli log-likelihood, which is equivalent to minimizing the binary cross-entropy (log loss) that most binary classifiers are trained on. A/B tests treat each visitor's conversion as a Bernoulli outcome and compare two success probabilities; spam filters, click-through-rate models and diagnostic-test analyses do the same. The Bernoulli parameter also has a natural Bayesian treatment through the beta distribution, which is why conversion-rate experiments are so often analyzed with beta-Bernoulli updating.

The usual pitfalls concern the word independent and the idea that p is known. If trials influence each other (customers who see the same ad in a shared household, repeated measurements on one patient) or p drifts over time, the sum is no longer binomial and standard errors computed as if it were will be too small. Another confusion is treating a single Bernoulli outcome as evidence about p: one flip landing heads says almost nothing about whether the coin is fair, and the probability of a 0/1 result is a property of the process, not of any one observed 0 or 1. Finally, a predicted probability of 0.9 does not mean that case will be 0.9 successful; each case is still all-or-nothing, and the probability only describes the long-run frequency among many such cases.

Read the Bernoulli distribution as the one-parameter atom of randomness: a single yes/no trial whose entire behavior (mean p, variance p(1 − p)) is set by one probability, and from which binomial counts, geometric waiting times, logistic regression and every conversion-rate experiment are assembled.