Polynomial Features is a feature engineering technique that creates new features by computing polynomial terms (squares, cubes) and interaction terms (products of features) from existing variables — enabling linear models to learn non-linear decision boundaries by expanding the feature space from $[a, b]$ to $[1, a, b, a^2, ab, b^2]$, where the interaction term $ab$ can capture relationships that neither $a$ nor $b$ reveals alone (house price depends on length × width = area, not length or width independently).

What Are Polynomial Features?

• Definition: A transformation that generates new features by computing all polynomial combinations of input features up to a specified degree — including squared terms ($a^2$), interaction terms ($ab$), and higher-order combinations ($a^2b$, $ab^2$).
• Why?: A linear model can only learn $y = w_1a + w_2b + bias$ — a flat plane in 3D. By adding $a^2$, $ab$, and $b^2$ as features, the same linear model now fits $y = w_1a + w_2b + w_3a^2 + w_4ab + w_5b^2 + bias$ — a curved surface that captures non-linear patterns.
• Key Insight: The model remains "linear" in its parameters (it's still a weighted sum) but non-linear in its features — this is the power of feature engineering.

Polynomial Expansion Example

Starting with features $a$ and $b$:

Interaction Terms: The Most Valuable Component

Python Implementation

from sklearn.preprocessing import PolynomialFeatures

poly = PolynomialFeatures(degree=2, include_bias=False,
                          interaction_only=False)
X_poly = poly.fit_transform(X)  # [a, b] -> [a, b, a², ab, b²]

# Interaction only (no squared terms)
inter = PolynomialFeatures(degree=2, interaction_only=True,
                           include_bias=False)
X_inter = inter.fit_transform(X)  # [a, b] -> [a, b, ab]

The Dimensionality Explosion Problem

Solutions: Use interaction_only=True (skip squared terms), apply feature selection after expansion, or use regularization (Ridge/Lasso) to zero out unimportant terms.

When to Use Polynomial Features

Polynomial Features is the feature engineering technique that gives linear models non-linear power — creating squared and interaction terms that enable linear regression and logistic regression to fit curved decision boundaries, with the critical caveat that dimensionality grows combinatorially and regularization is essential to prevent overfitting on the expanded feature set.