neural network gaussian process

**NNGP** (Neural Network Gaussian Process) is a **theoretical result showing that infinitely wide neural networks with random weights converge to Gaussian Processes** — the distribution over functions defined by the random initialization becomes exactly a GP in the infinite-width limit. **What Is NNGP?** - **Result**: A single hidden-layer network with $n ightarrow infty$ neurons and random weights defines a GP with a specific kernel. - **Kernel**: The NNGP kernel is determined by the activation function and the weight/bias distributions. - **Deep Networks**: Each layer's GP kernel is defined recursively from the previous layer. - **Papers**: Neal (1996), Lee et al. (2018), Matthews et al. (2018). **Why It Matters** - **Bayesian DL**: Provides exact Bayesian inference for infinitely wide networks (no MCMC needed). - **Uncertainty**: Inherits GP's calibrated uncertainty estimates. - **Theory**: Connects deep learning to the well-understood GP framework, enabling analytical results. **NNGP** is **the bridge between neural networks and Gaussian Processes** — revealing that infinitely wide random networks are, mathematically, just kernel machines.