Statistical Sample

A sample is a subset of a population selected for actual measurement, used to estimate characteristics of the full population when measuring every single member directly isn't feasible. The entire logic of inferential statistics depends on one condition being true about how that sample was obtained: it has to represent the population it's drawn from reasonably well, because a statistic computed from an unrepresentative sample doesn't estimate the population's true characteristic no matter how large the sample is or how sophisticated the analysis applied to it afterward.

Sampling methods split broadly into probability sampling, where every member of the population has a known, nonzero chance of being selected, and non-probability sampling, where that guarantee doesn't hold — and only probability sampling provides a rigorous mathematical basis for generalizing a sample statistic back to the population with a quantifiable degree of confidence. Simple random sampling, the most basic probability method, gives every member an equal chance of selection, analogous to drawing names from a hat. Stratified sampling divides the population into meaningful subgroups (strata) first — by age group, by region, by customer tier — then samples randomly within each stratum, which guarantees proportional representation of each subgroup rather than leaving that representation to chance, a meaningful advantage when a subgroup is small enough that pure random sampling might undersample or even miss it entirely by luck alone. Cluster sampling instead randomly selects whole pre-existing groups (clusters) — entire stores, entire classrooms, entire geographic regions — and measures every member within the selected clusters, a method chosen primarily for practical cost reasons when individually sampling across a dispersed population would be logistically difficult or expensive.

Convenience sampling — selecting whoever is easiest to reach, like surveying people who happen to walk past a particular location, or analyzing only the customers who happen to have left a review — is the most common non-probability method precisely because it's the cheapest and fastest to run, but it carries the highest risk of selection bias, since "easy to reach" or "likely to leave a review" is rarely independent of the characteristic actually being measured; customers who leave reviews, for instance, tend to be systematically different (often more extreme in either satisfaction or dissatisfaction) than the full customer base, which is exactly the kind of distortion that makes a convenience sample a poor stand-in for the population it's meant to represent.

Sample size interacts with representativeness in a way that's frequently misunderstood: a larger sample reduces the margin of error around an estimate — the random, chance-driven uncertainty from having sampled rather than measured everyone — but does nothing to fix a biased sampling method, since bias is a systematic distortion in who or what got selected, not a matter of not having selected enough. A sample of a million unrepresentative respondents is not more trustworthy than a sample of a thousand properly randomized ones for estimating a population characteristic; it's simply a larger, equally biased sample, which is the precise lesson the 1936 Literary Digest poll's massive but systematically skewed sample is most often cited to illustrate.

Sample size calculation — determining in advance how large a sample needs to be to detect a meaningful effect with a desired level of statistical confidence — is a standard step in rigorous experimental design (particularly in A/B testing and clinical trials), balancing the cost of collecting more data against the risk of running an underpowered study too small to reliably detect a real effect even if one genuinely exists.

Read a sample as a deliberately chosen stand-in for a population too large or costly to measure directly, and recognize that how it was chosen matters more than how large it is: probability sampling methods — simple random, stratified, cluster — are what give a sample statistic a rigorous mathematical basis for generalizing back to the population, while non-probability methods like convenience sampling trade that rigor for ease of collection, and no amount of additional sample size can correct a sampling method that was systematically biased from the start.