tukey biweight
**Tukey's biweight loss** is an **M-estimator loss function that completely and absolutely ignores errors exceeding a threshold** — providing hard outlier rejection where the gradient vanishes for extreme deviations, enabling models to learn data patterns despite massive contamination from gross errors, the ultimate robustness for filtering erroneous data.
**What Is Tukey's Biweight Loss?**
Tukey's biweight (also called bisquare) is a redescending M-estimator from robust statistics that behaves like a quadratic penalty near zero, gradually decreases in influence for moderate errors, and completely rejects (zero gradient) for large errors beyond threshold c. This is the ultimate form of outlier rejection — unlike Huber and Cauchy where large errors still contribute some gradient, Tukey completely ignores them.
**Mathematical Definition**
Tukey biweight loss:
```
ρ(x) =
(c²/6) * [1 - (1 - (x/c)²)³] if |x| ≤ c (influence region)
c²/6 if |x| > c (rejection region)
Weight function w(x) = (1 - (x/c)²)² if |x| ≤ c, else 0
Gradient: ∂ρ/∂x = x * (1 - (x/c)²)² if |x| ≤ c, else 0
```
Three distinct regions:
1. **|x| < c**: Quadratic-like behavior with influence gradually decreasing
2. **|x| = c**: Transition point where influence reaches maximum
3. **|x| > c**: Gradient exactly zero — complete outlier rejection
**Why Tukey's Biweight Matters**
- **Hard Rejection**: Errors beyond threshold completely ignored — maximum possible robustness
- **Redescending Property**: Influence increases then decreases with error magnitude
- **Classical Foundation**: Developed by John Tukey, proven robust statistics researcher
- **RANSAC-Like**: Functions similar to RANSAC consensus but through soft downweighting
- **Parameter Control**: Threshold c allows tuning how to classify outliers
- **Justifiable**: Works even with 50%+ contamination (breakdown point = 0.5)
**The Redescending Property**
Unlike Huber and Cauchy where influence monotonically increases, Tukey's biweight reaches maximum influence at error = 0.3c, then decreases, reaching zero at c:
```
Influence vs Error Magnitude:
|
| ╱╲
| ╱ ╲
| ╱ ╲___
| ╱ (zero influence beyond c)
|___________|____
0 c
```
**Comparison: Outlier Rejection Approaches**
| Error = 5c | MSE | Huber | Cauchy | Tukey |
|-----------|-----|-------|--------|-------|
| Loss | (5c)² = 25c² | 5c * c = 5c² | c² ln(26) ≈ 3.3c² | c²/6 ≈ 0.167c² |
| Influence | Extreme | High | Moderate | Zero |
| Gradient Magnitude | 10c | c | Small | Exactly 0 |
**Parameter Selection**
- **c = 1.0**: Standard default
- **c = 4.685 * σ**: Recommended for Gaussian noise with std σ (breakdown point 50%)
- **Strategy for tuning**:
- Compute residual median absolute deviation (MAD)
- Set c = 4.685 * MAD
- Or cross-validate on validation set
**Implementation**
PyTorch:
```python
def tukey_biweight_loss(predictions, targets, c=1.0):
errors = (predictions - targets)
mask = (errors.abs() <= c).float()
term = 1 - (errors / c) ** 2
loss = (c**2 / 6) * mask * (1 - term ** 3)
return loss.mean()
```
NumPy (for offline analysis):
```python
import numpy as np
def tukey_biweight(x, c=1.0):
mask = np.abs(x) <= c
loss = np.zeros_like(x, dtype=float)
loss[mask] = (c**2/6) * (1 - (1 - (x[mask]/c)**2)**3)
loss[~mask] = c**2/6
return loss
```
**When to Use Tukey's Biweight**
- **Gross Outliers**: Data contains obviously wrong values (sensor failures, data entry errors)
- **Contaminated Data**: Unknown large percentage of corrupted observations
- **Automatic Outlier Detection**: Threshold enables identifying rejected samples
- **Robust Fitting**: Least squares fitting that ignores bad leverage points
- **Certified Protection**: 50% breakdown point guarantees robustness
- **High-Dimensional**: More robust than alternatives in high-dimensional settings
**Practical Applications**
**Robust Least Squares**: Fitting lines, planes, curves to data with gross errors — automatic leverage point rejection enables fitting despite bad measurements.
**Astronomical Data**: Detecting planets from stellar brightness where cosmic rays and instrumental glitches contaminate significant portion of measurements; Tukey enables using all data while ignoring artifact-corrupted observations.
**Survey Data**: Statistical analysis of survey responses with occasional fraudulent/nonsense entries; Tukey automatically downweights or ignores impossible values without manual cleaning.
**Geospatial Analysis**: GPS trajectories with occasional wild spikes (multipath, jamming); Tukey filters outlier positions while preserving real movements.
**Quality Control**: Manufacturing processes flagging and ignoring equipment malfunctions while maintaining statistical model of normal operations.
Tukey's biweight is **the maximum-robustness outlier elimination** — hard rejection for gross errors enables learning from contaminated data that would destroy other methods, providing theoretical guarantee of robustness even with 50% contamination.