activation maximization

**Activation Maximization** is the **optimization-based approach to generating inputs that maximally activate a target neuron or output class in a neural network** — using gradient ascent in input space to find (or synthesize) the input pattern that a neuron responds most strongly to. **Activation Maximization Process** - **Target**: Choose a neuron, channel, layer, or output class to maximize. - **Initialize**: Start with noise, a fixed image, or a learned prior (generator network). - **Gradient Ascent**: Compute $ abla_x a_{target}(x)$ and update the input: $x leftarrow x + eta abla_x a_{target}$. - **Regularization**: Apply image priors (total variation, frequency penalization, learned priors) to produce natural-looking results. **Why It Matters** - **Neuron Identity**: Reveals the "ideal stimulus" for each neuron — what it has learned to represent. - **Class Visualization**: Generate the "ideal" input for each output class — the network's prototype of each category. - **GAN Priors**: Using a GAN generator as the parameterization produces photorealistic activation maximization. **Activation Maximization** is **finding the neuron's favorite input** — the optimization-based core technique behind feature visualization and neural network understanding.