network morphism

**Network Morphism** is a **technique for transforming a trained neural network into a larger or differently structured network** — while preserving its learned function exactly, allowing the new network to continue training from a warm start rather than from random initialization. **What Is Network Morphism?** - **Definition**: Function-preserving transformations on neural networks. - **Operations**: - **Widen**: Add more neurons/filters to a layer (pad with zeros). - **Deepen**: Insert a new identity layer (initialized as pass-through). - **Reshape**: Change kernel size while preserving learned features. - **Guarantee**: $f_{new}(x) = f_{old}(x)$ for all inputs immediately after morphism. **Why It Matters** - **NAS (Neural Architecture Search)**: Efficiently explore architectures by morphing one into another without retraining from scratch. - **Transfer Learning**: Grow a small model into a larger one if more capacity is needed. - **Curriculum**: Start small, grow as data or task complexity increases. **Network Morphism** is **neural evolution** — growing neural networks organically like biological brains rather than rebuilding them from scratch.