parseval networks

**Parseval Networks** are **neural networks whose weight matrices are constrained to have spectral norm ≤ 1 using Parseval tight frame constraints** — ensuring each layer is a contraction, resulting in a globally Lipschitz-constrained network with improved robustness. **How Parseval Networks Work** - **Parseval Tight Frame**: Weight matrices satisfy $WW^T = I$ (when the matrix is wide) or $W^TW = I$ (when tall). - **Regularization**: Add a regularization term $eta |WW^T - I|^2$ to the training loss. - **Projection**: Periodically project weights onto the set of tight frames during training. - **Convex Combination**: Blend the projected weights with current weights: $W leftarrow (1+eta)W - eta WW^TW$. **Why It Matters** - **Lipschitz-1**: Each layer is a contraction — the full network has Lipschitz constant ≤ 1. - **Adversarial Robustness**: Parseval networks show improved robustness to adversarial perturbations. - **Theoretical Foundation**: Grounded in frame theory from signal processing. **Parseval Networks** are **contraction-constrained architectures** — using tight frame theory to ensure each layer contracts rather than amplifies perturbations.