simam

**SimAM** (Simple Parameter-Free Attention Module) is a **3D attention mechanism that generates weights for each neuron without any learnable parameters** — using energy-based neuroscience principles to estimate each neuron's importance based on its distinctiveness from surrounding neurons. **How Does SimAM Work?** - **Energy Function**: $e_t = frac{1}{M-1}sum_{i=1}^{M-1}(-1-(x_t - x_i)^2)^2 + (1-hat{x}_t)^2$ per neuron. - **Importance**: Neurons with lower energy (more distinct from neighbors) get higher attention weights. - **3D Attention**: Produces per-neuron weights across all three dimensions (C, H, W) simultaneously. - **No Parameters**: Entirely computed from the feature values — zero learnable parameters. - **Paper**: Yang et al. (2021). **Why It Matters** - **Parameter-Free**: No additional parameters to train — attention is purely computed from input statistics. - **Neuroscience-Inspired**: Based on the visual neuroscience concept of neuronal spatial suppression. - **Unified**: Simultaneously provides channel and spatial attention in a single mechanism. **SimAM** is **parameter-free 3D attention** — using neuroscience-inspired energy functions to assess each neuron's importance without learning a single extra weight.