polynomial
**Polynomial Features** is a **feature engineering technique that creates new features by computing polynomial terms (squares, cubes) and interaction terms (products of features) from existing variables** — enabling linear models to learn non-linear decision boundaries by expanding the feature space from $[a, b]$ to $[1, a, b, a^2, ab, b^2]$, where the interaction term $ab$ can capture relationships that neither $a$ nor $b$ reveals alone (house price depends on length × width = area, not length or width independently).
**What Are Polynomial Features?**
- **Definition**: A transformation that generates new features by computing all polynomial combinations of input features up to a specified degree — including squared terms ($a^2$), interaction terms ($ab$), and higher-order combinations ($a^2b$, $ab^2$).
- **Why?**: A linear model can only learn $y = w_1a + w_2b + bias$ — a flat plane in 3D. By adding $a^2$, $ab$, and $b^2$ as features, the same linear model now fits $y = w_1a + w_2b + w_3a^2 + w_4ab + w_5b^2 + bias$ — a curved surface that captures non-linear patterns.
- **Key Insight**: The model remains "linear" in its parameters (it's still a weighted sum) but non-linear in its features — this is the power of feature engineering.
**Polynomial Expansion Example**
Starting with features $a$ and $b$:
| Degree | Generated Features | Count |
|--------|-------------------|-------|
| 1 | $a, b$ | 2 |
| 2 | $a, b, a^2, ab, b^2$ | 5 |
| 3 | $a, b, a^2, ab, b^2, a^3, a^2b, ab^2, b^3$ | 9 |
| d | All combinations up to degree d | Rapidly grows |
**Interaction Terms: The Most Valuable Component**
| Features | Interaction | Real-World Meaning |
|----------|------------|-------------------|
| Length, Width | Length × Width | Area (determines house price) |
| Education, Experience | Education × Experience | Combined effect on salary |
| Temperature, Humidity | Temp × Humidity | Feels-like / heat index |
| Ad Spend, Season | Spend × Season | Holiday ad effectiveness |
**Python Implementation**
```python
from sklearn.preprocessing import PolynomialFeatures
poly = PolynomialFeatures(degree=2, include_bias=False,
interaction_only=False)
X_poly = poly.fit_transform(X) # [a, b] -> [a, b, a², ab, b²]
# Interaction only (no squared terms)
inter = PolynomialFeatures(degree=2, interaction_only=True,
include_bias=False)
X_inter = inter.fit_transform(X) # [a, b] -> [a, b, ab]
```
**The Dimensionality Explosion Problem**
| Original Features | Degree | New Features | Growth |
|------------------|--------|-------------|--------|
| 2 | 2 | 5 | 2.5× |
| 10 | 2 | 65 | 6.5× |
| 50 | 2 | 1,325 | 26.5× |
| 100 | 2 | 5,150 | 51.5× |
| 100 | 3 | 176,850 | 1,768× |
**Solutions**: Use `interaction_only=True` (skip squared terms), apply feature selection after expansion, or use regularization (Ridge/Lasso) to zero out unimportant terms.
**When to Use Polynomial Features**
| Use | Don't Use |
|-----|----------|
| Linear models with non-linear patterns | Tree-based models (they capture interactions natively) |
| Known feature interactions (area = L × W) | Very high-dimensional data (dimensionality explodes) |
| Small number of features (<20) | When you already have hundreds of features |
| Paired with regularization (Ridge/Lasso) | Without regularization (severe overfitting) |
**Polynomial Features is the feature engineering technique that gives linear models non-linear power** — creating squared and interaction terms that enable linear regression and logistic regression to fit curved decision boundaries, with the critical caveat that dimensionality grows combinatorially and regularization is essential to prevent overfitting on the expanded feature set.