orthogonal convolutions

**Orthogonal Convolutions** are **convolutional layers with orthogonality constraints on the kernel matrices** — ensuring that the convolutional transformation preserves the norm of feature maps, resulting in a layer-wise Lipschitz constant of exactly 1. **Implementing Orthogonal Convolutions** - **Cayley Transform**: Parameterize the convolution kernel using the Cayley transform of a skew-symmetric matrix. - **Björck Orthogonalization**: Iteratively project weight matrices toward orthogonality during training. - **Block Convolution**: Reshape the convolution into a matrix operation and enforce orthogonality on the matrix. - **Householder Parameterization**: Compose Householder reflections to build orthogonal transformations. **Why It Matters** - **Exact Lipschitz**: Each orthogonal layer has Lipschitz constant exactly 1 — the full network's Lipschitz constant equals 1. - **No Signal Loss**: Orthogonal layers preserve feature map norms — no vanishing or exploding signals. - **Certifiable**: Networks with orthogonal convolutions have tight, easily computable robustness certificates. **Orthogonal Convolutions** are **norm-preserving feature extractors** — convolutional layers that maintain exact Lipschitz-1 behavior for provably robust networks.