Feature Visualization is a technique that generates synthetic input images that maximally activate specific neurons, channels, or layers in a neural network — revealing what features the network has learned to detect at each level of abstraction.

How Feature Visualization Works

• Objective: $x^* = argmax_x a_k(x) - lambda R(x)$ where $a_k$ is the target neuron activation and $R$ is a regularizer.
• Optimization: Start from noise or a random image and iteratively optimize via gradient ascent.
• Regularization: Total variation, Gaussian blur, jitter, and transformation robustness prevent adversarial noise.
• Diversity: Generate multiple visualizations per neuron using diversity objectives for richer understanding.

Why It Matters

• Layer Hierarchy: Low layers detect edges/textures, mid layers detect parts/patterns, high layers detect objects/concepts.
• Debugging: Reveals spurious features (e.g., watermarks, background correlations) the model relies on.
• Communication: Beautiful, intuitive visualizations that communicate network behavior to non-experts.

Feature Visualization is asking the network to dream — generating synthetic inputs that reveal what patterns each neuron has learned to recognize.