Chi-Square Distribution

The chi-square distribution is the distribution of a sum of squared independent standard normal variables: if Z1 through Zk are independent standard normals, then Z1² + Z2² + ... + Zk² follows a chi-square distribution with k degrees of freedom. Because a square cannot be negative, it lives only on positive values, is right-skewed (sharply so for small k) and has mean k and variance 2k, drifting toward a symmetric bell as k grows. It is the sampling distribution of many measures of total squared discrepancy, and its place in statistics rests on Karl Pearson's 1900 paper in the Philosophical Magazine, which introduced the chi-square criterion for judging whether observed counts are plausibly consistent with a hypothesized pattern.

Pearson's insight was a single statistic that sums the standardized squared gaps between observed and expected counts: the sum, over all categories, of (observed − expected)² divided by expected. If the hypothesized model is right and the expected counts are not tiny, this quantity follows an approximate chi-square distribution, so a value far into the upper tail signals data that fit the model too poorly to be chance. The same statistic drives the two classic categorical tests: the goodness-of-fit test, which asks whether counts match a specified distribution (a die's faces, a Mendelian ratio, a claimed market share), and the test of independence in a contingency table, which asks whether two categorical variables are related, with expected counts computed from the row and column totals under independence.

Degrees of freedom count how many of the squared discrepancies are free to vary, and getting them right was historically contentious. For a goodness-of-fit test with c categories and a fully specified model the degrees of freedom are c − 1, because the counts must sum to the total; for a test of independence in a table with r rows and c columns they are (r − 1) × (c − 1). Pearson's original paper used the wrong degrees of freedom when parameters had to be estimated from the data, and Ronald Fisher's work in 1922 (and further papers shortly after) showed that each estimated parameter removes one more degree of freedom, a correction now standard. The 5% critical values illustrate the scale: 3.841 for 1 degree of freedom (which is exactly 1.96², the link back to the standard-normal-distribution), 5.991 for 2, 9.488 for 4 and 18.307 for 10.

A second family of uses comes from the normal sample variance: when data are drawn from a normal population with variance σ², the quantity (n − 1) × s² / σ² is chi-square with n − 1 degrees of freedom. That fact yields confidence intervals and tests for a variance or standard deviation, which matter in manufacturing tolerances and measurement precision, and it is the denominator ingredient that, combined with a standard normal, defines Student's t (see student-t-distribution). In applied work, chi-square tests screen categorical features before modeling, check whether a randomized experiment's arms are balanced on categories, test the fit of count models, and flag survey weights that don't match known demographics, while likelihood-ratio tests in generalized linear models rely on the same distribution asymptotically.

The main pitfalls are about what a significant result does and does not say. A chi-square test applies to counts, not percentages, and the approximation is poor when expected counts are very small (a common rule of thumb asks for expected counts of about 5 or more per cell, with Fisher's exact test as the alternative). Tests of independence say nothing about the strength of association: with enough data, a trivially small deviation gives a minuscule p-value, so effect-size measures such as Cramér's V should accompany it, and a significant result never reveals which cells drive the difference without checking residuals. Finally, failing to reject is not proof of fit, and the test requires independent observations; repeated measurements on the same units violate that and inflate the statistic.

Read the chi-square distribution as the sampling law of total squared discrepancy: built from squared standard normals, it converts the gap between observed and expected counts (Pearson, 1900) into a single number whose tail probability says whether the pattern is plausibly chance, with degrees of freedom tracking how many gaps are truly free to vary.