Categorical Data

Categorical data is data that assigns each observation to one of a fixed, finite set of groups rather than measuring a quantity — a customer's country, a product's color, a transaction's payment method. The defining test is whether arithmetic on the values is meaningful: it isn't, even when a category happens to be stored as a number (a customer segment coded 1, 2, 3 is still categorical, and averaging those codes produces a number with no real interpretation). This is the mirror case of numerical data, and the two together cover essentially every variable a dataset can contain, which is why correctly identifying which one a given column actually is — not just how it happens to be stored — is one of the first and most consequential decisions in any analysis.

Categorical data splits into nominal and ordinal, and the split determines what comparisons between categories are valid. Nominal categories have no inherent order — color, country, payment method — so the only meaningful operations are counting and checking equality: how many purchases were made by credit card versus PayPal, which category occurs most often (the mode). Ordinal categories have a meaningful order but no fixed, equal distance between adjacent values — a satisfaction rating of "poor," "fair," "good," "excellent" is ordered, but the gap between "poor" and "fair" isn't necessarily the same size as the gap between "good" and "excellent," which is why computing a mean on ordinal data (common in practice, treating a 1-5 star rating as if it were numerical) is a widely used but statistically imprecise shortcut — technically defensible as an approximation in large samples, but not a rigorous use of the scale.

Encoding categorical data into a numeric form a machine learning model can use is where the nominal-versus-ordinal distinction has its most direct practical consequence. Nominal data is typically one-hot encoded: each category becomes its own binary column (is_credit_card, is_paypal, is_bank_transfer), preventing the model from inferring a false order or distance between categories that genuinely have none. Ordinal data can instead use ordinal encoding, mapping categories to integers in their natural order (poor=1, fair=2, good=3, excellent=4), which preserves the real ordering information that one-hot encoding would discard. Choosing one-hot encoding for a genuinely ordinal variable throws away real signal; choosing ordinal encoding for a genuinely nominal variable introduces a false signal — both are common, specific, and avoidable modeling mistakes.

High-cardinality categorical data — a categorical variable with a very large number of distinct categories, like zip code, product SKU, or user ID — creates a practical problem that low-cardinality categorical data (like gender or a handful of product tiers) doesn't: one-hot encoding a high-cardinality variable can expand a dataset from a handful of columns to thousands, degrading model performance and training speed. Techniques developed specifically for this problem include target encoding (replacing each category with the average outcome for that category, computed carefully to avoid leaking the outcome into the feature) and embedding layers (used in deep learning, learning a dense low-dimensional vector representation for each category rather than a sparse one-hot vector) — both exist because naive encoding stops scaling once category count gets large enough.

Statistical tests for categorical data rely on a different family of methods than numerical data does, most commonly the chi-square test, which checks whether the distribution of one categorical variable differs significantly across the groups of another — whether purchase category differs by region, for instance — by comparing observed category counts against the counts expected if the two variables were independent. This is the categorical-data analog to a t-test or ANOVA for numerical data: a different mathematical machine built specifically for comparing group frequencies rather than group means, and applying a mean-comparison test to categorical data (or a frequency-comparison test to numerical data) is a basic but real class of statistical misuse.

Read categorical data as the class of data whose value is a label, not a measurement: nominal-versus-ordinal decides whether order and distance carry real meaning, that distinction directly drives which encoding scheme is correct for modeling, and the field's dedicated tools for this data type — one-hot and target encoding, the chi-square test — all exist because the numerical toolkit built around means and variances simply doesn't apply to data whose values were never meant to be added together.