nfnet

**NFNet** (Normalizer-Free Networks) is a **high-performance CNN architecture that achieves state-of-the-art accuracy without using batch normalization** — using Adaptive Gradient Clipping (AGC) and carefully designed signal propagation to replace BatchNorm entirely. **What Is NFNet?** - **No BatchNorm**: Eliminates all BN layers. Uses Scaled Weight Standardization + AGC instead. - **AGC**: Clips gradients based on the ratio of gradient norm to parameter norm (unit-wise). - **Signal Propagation**: Carefully designed variance-preserving residual connections using a scaling factor. - **Paper**: Brock et al. (2021). **Why It Matters** - **SOTA Without BN**: NFNet-F1 achieves 86.5% ImageNet top-1 (SOTA at time of release) without any normalization. - **Large Batch Friendly**: No BN -> no batch size dependency -> cleaner distributed training. - **Simplicity**: Removes the BN dependency that complicates training, transfer learning, and inference. **NFNet** is **the proof that BatchNorm is optional** — achieving record accuracy by replacing normalization with principled gradient clipping and signal propagation.