welsch loss

**Welsch Loss** is a **robust loss function that bounds the maximum penalty for outliers** — using an exponential form $L(r) = frac{c^2}{2}[1 - exp(-(r/c)^2)]$ that asymptotes to a constant for large residuals, preventing outliers from dominating the optimization. **Welsch Loss Properties** - **Form**: $L(r) = frac{c^2}{2}[1 - exp(-r^2/c^2)]$ — converges to $c^2/2$ as $|r| ightarrow infty$. - **Small Residuals**: Behaves like squared loss for $|r| ll c$ — standard quadratic behavior. - **Large Residuals**: Loss saturates at $c^2/2$ — outliers have bounded, constant influence. - **Parameter $c$**: Controls the transition between quadratic and constant regions (inlier-outlier threshold). **Why It Matters** - **Robust Regression**: Completely eliminates the influence of extreme outliers — they can't dominate the loss. - **Process Data**: Semiconductor process data often contains outliers from sensor failures — Welsch loss prevents corruption. - **Smooth**: Unlike Huber loss (which has a slope change at the threshold), Welsch loss is infinitely smooth. **Welsch Loss** is **the gentlest robust loss** — smoothly transitioning from quadratic to bounded behavior for complete outlier immunity.