feature visualization

**Feature Visualization** is a **technique that generates synthetic input images that maximally activate specific neurons, channels, or layers in a neural network** — revealing what features the network has learned to detect at each level of abstraction. **How Feature Visualization Works** - **Objective**: $x^* = argmax_x a_k(x) - lambda R(x)$ where $a_k$ is the target neuron activation and $R$ is a regularizer. - **Optimization**: Start from noise or a random image and iteratively optimize via gradient ascent. - **Regularization**: Total variation, Gaussian blur, jitter, and transformation robustness prevent adversarial noise. - **Diversity**: Generate multiple visualizations per neuron using diversity objectives for richer understanding. **Why It Matters** - **Layer Hierarchy**: Low layers detect edges/textures, mid layers detect parts/patterns, high layers detect objects/concepts. - **Debugging**: Reveals spurious features (e.g., watermarks, background correlations) the model relies on. - **Communication**: Beautiful, intuitive visualizations that communicate network behavior to non-experts. **Feature Visualization** is **asking the network to dream** — generating synthetic inputs that reveal what patterns each neuron has learned to recognize.