Aggregation

Aggregation is the operation of combining many individual values into a single summary number — summing, averaging, counting, finding the minimum or maximum — and it's the "apply" step of the split-apply-combine pattern that grouping implements: once a dataset is split into groups, aggregation is what actually computes a result for each one. Aggregation can also run without any grouping at all, collapsing an entire column into one overall summary (total revenue across every order, average order size across the whole dataset), which is the simplest and most common first question asked of almost any new dataset.

The choice of which aggregate function to use is not a neutral or interchangeable decision, because different aggregates answer meaningfully different questions and can tell very different stories about the same underlying data. Sum answers "how much in total," useful for additive quantities like revenue or units sold but meaningless for something like a rating or percentage. Count answers "how many," distinct from sum in that it counts occurrences rather than totaling a value, and count of distinct values (a separate, commonly needed variant) answers "how many unique," which is a different and often more useful number than a raw count when duplicates are possible. Average answers "what's typical," but is sensitive to outliers in a way the median — a different aggregate entirely — is not, which is why choosing mean versus median as the "typical value" aggregate is itself a meaningful analytical decision rather than a stylistic preference, especially on skewed data like income or order size, where a handful of extreme values can pull the mean substantially away from what most individual data points actually look like.

Aggregating across groups of unequal size is a specific, recurring source of misleading conclusions known as Simpson's paradox, a statistical phenomenon where a trend that appears in several different groups reverses or disappears when those groups are aggregated together — famously documented in a 1973 study of UC Berkeley graduate admissions, where men appeared to be admitted at a higher overall rate than women across the university as a whole, but when broken down by individual department, women were admitted at an equal or higher rate in most departments, the reversal explained by women disproportionately applying to more competitive departments with lower admission rates overall. Simpson's paradox is the clearest demonstration of why an aggregated number can be technically correct and still actively misleading about the underlying pattern, which is why examining a result at both the aggregate level and broken down by relevant subgroups is standard practice before drawing a causal or comparative conclusion from an aggregated statistic.

Pre-aggregated data — summary tables computed and stored once, in advance, rather than recalculated from raw transaction-level data on every query — is a standard performance optimization in analytics infrastructure (commonly implemented as materialized views or dedicated summary tables), trading storage space and some staleness risk for significantly faster query response, since summing a pre-computed daily total is far cheaper than re-summing millions of individual transaction rows every single time a dashboard loads. This optimization comes with a real tradeoff worth naming explicitly: pre-aggregated data can only answer the specific questions it was pre-aggregated to answer, and any analysis needing a different breakdown than what was pre-computed has to fall back to the slower, raw-data calculation — which is why analytics teams generally keep both the pre-aggregated summary tables for common dashboard queries and access to the full raw data for ad hoc, non-standard analysis.

Read aggregation as the step that turns many individual observations into one number meant to represent them, and recognize that the choice of which aggregate function to use is itself an analytical decision with real consequences: sum, count, mean, and median each tell a different kind of story even applied to the exact same underlying data, Simpson's paradox is the sharpest illustration of how an aggregated number can actively mislead relative to what's happening within the underlying subgroups, and the practical infrastructure built around aggregation — pre-computed summary tables — trades flexibility for speed in a tradeoff that has to be managed deliberately rather than assumed away.