lipschitz constant estimation

**Lipschitz Constant Estimation** is the **computation or bounding of a neural network's Lipschitz constant** — the maximum ratio of output change to input change, $|f(x_1) - f(x_2)| leq L |x_1 - x_2|$, measuring the network's maximum sensitivity to input perturbations. **Estimation Methods** - **Naive Bound**: Product of weight matrix operator norms across layers — fast but often very loose. - **SDP Relaxation**: Semidefinite programming relaxation for tighter bounds (LipSDP). - **Sampling-Based**: Estimate a lower bound by sampling many input pairs and computing maximum slope. - **Layer-Peeling**: Tighter compositional bounds that exploit network structure. **Why It Matters** - **Robustness Certificate**: $L$ directly gives the maximum prediction change for any $epsilon$-perturbation: $Delta f leq L epsilon$. - **Sensitivity**: Small Lipschitz constant = stable, robust model. Large = potentially sensitive and fragile. - **Regularization**: Training to minimize $L$ (Lipschitz regularization) directly improves adversarial robustness. **Lipschitz Estimation** is **measuring maximum sensitivity** — bounding how much the network's output can change for a given input perturbation.