spectral normalization

**Spectral Normalization** is a **weight normalization technique that constrains the spectral norm (largest singular value) of each weight matrix to 1** — enforcing a 1-Lipschitz constraint on the layer, which stabilizes GAN discriminator training without gradient penalty's computational cost. **How Does Spectral Normalization Work?** - **Normalization**: $ar{W} = W / sigma(W)$ where $sigma(W)$ is the largest singular value of $W$. - **Power Iteration**: $sigma(W)$ is estimated efficiently using one step of power iteration per training step. - **Cost**: Negligible — one matrix-vector multiply per layer per step. - **Paper**: Miyato et al. (2018). **Why It Matters** - **GAN Stability**: Stabilizes discriminator training without the per-sample cost of gradient penalty. - **Efficiency**: Much cheaper than WGAN-GP (which requires gradient computation through the discriminator). - **Universal**: Applied in BigGAN, StyleGAN, and most modern GANs as a default technique. **Spectral Normalization** is **the singular value leash** — keeping each layer's transformation gentle enough to produce stable, high-quality GAN training.