Probability Mass Function

A probability mass function (pmf) is a function that gives, for each possible value of a discrete random variable, the probability that the variable takes exactly that value. Written p(x) = P(X = x), it assigns a non-negative number to every value the variable can take, and these numbers must add up to exactly 1. For one roll of a fair die the pmf is 1/6 at each of the values 1 through 6; for a biased coin that lands heads with probability 0.3, it is 0.3 at heads (coded 1) and 0.7 at tails (coded 0). The pmf is the complete description of a discrete distribution: once it is known, the probability of any event, and every summary such as the mean and variance, can be computed from it.

The word "mass" is deliberate: probability is concentrated in lumps at specific points, like physical mass attached to discrete positions. A pmf is drawn as a bar chart or a set of vertical spikes, and the height of each bar is a real probability. This is what separates it from the probability density function used for continuous variables, where the height at a point is not a probability and any single exact value has probability zero. For discrete variables, the probability of any set of values is found by simply adding the corresponding bars, so P(X is 7 or 11) for the sum of two dice is 6/36 + 2/36 = 8/36. Related is the cumulative distribution function, which stacks the bars from the left, and the pmf can be recovered from it by subtracting successive values.

The sum of two dice is the standard worked example of building a pmf by counting. There are 36 equally likely ordered outcomes, and the sum 2 arises only from (1,1), while 7 arises from six pairs, so the pmf rises linearly from 1/36 at 2 to 6/36 at 7 and falls symmetrically to 1/36 at 12. Nothing in the original dice is unequal; the unevenness comes entirely from how many ways each sum can be formed, which is why 7 is the most common total in craps and 2 and 12 are rare. Constructing a pmf typically follows this recipe: list the values the variable can take, count or compute the probability of each, and check that the total is 1. A failed total is a reliable sign that an outcome has been missed or double-counted.

A few named pmfs cover a large share of applied work. The Bernoulli pmf describes a single yes-or-no trial; the binomial counts successes across n independent trials, with probability given by the number of ways to choose k successes times p^k times (1 - p)^(n - k); the geometric counts trials until the first success; and the Poisson, introduced by Siméon Denis Poisson in 1837, models counts of rare events in a fixed interval, with p(k) = e^(-λ) × λ^k / k!. Ladislaus Bortkiewicz's 1898 study of Prussian cavalry deaths from horse kicks, "The Law of Small Numbers," is the classic demonstration that a Poisson pmf fits real count data. Today the same families model call arrivals, defects per unit, website visits per minute and mutation counts.

In machine learning, the pmf is the output object of every classifier. A softmax layer converts a network's raw scores into a pmf over the class labels, so the entries are non-negative and sum to 1, and a language model produces a pmf over its entire vocabulary for the next token. Training minimizes cross-entropy, which compares the predicted pmf with the observed one; entropy and Kullback-Leibler divergence, as defined by Shannon in 1948 and Kullback and Leibler in 1951, are all computed from pmfs. Likelihood-based fitting treats the pmf at the observed data as a function of its parameters, and maximizing it gives the maximum likelihood estimate.

The most common pitfalls involve confusing the pmf with its continuous cousin or with a histogram. A pmf value is a probability and can never exceed 1, whereas a density value can. An empirical histogram or relative-frequency table is an estimate of the pmf from data, not the pmf itself, so with small samples the bars can differ noticeably from the true values. Another mistake is to apply a pmf to a variable that is not truly discrete: measured quantities such as height are continuous, and a pmf would assign zero to every value. A final subtlety is that the support matters: p(x) is defined as zero for values outside the list of possibilities, and a legitimate pmf must check both non-negativity and the total of exactly 1 before it is used.

Read a probability mass function as the full list of a discrete variable's possible values with the probability sitting at each one: the bars are real probabilities that total 1, every event's probability is a sum of bars, and from this single table the mean, the spread and the entire behavior of the variable follow.