squeeze-and-excitation

**Squeeze-and-Excitation (SE)** is a **channel attention mechanism that adaptively recalibrates channel-wise feature responses** — by globally summarizing each channel (squeeze) and then learning inter-channel dependencies to produce per-channel importance weights (excitation). **How Does SE Work?** - **Squeeze**: Global average pooling across spatial dimensions: $z_c = frac{1}{HW}sum_{h,w} x_{c,h,w}$. - **Excitation**: Two FC layers with ReLU and sigmoid: $s = sigma(W_2 cdot ext{ReLU}(W_1 cdot z))$. - **Scale**: Multiply each channel by its learned importance: $hat{x}_c = s_c cdot x_c$. - **Paper**: Hu et al. (2018). Won ILSVRC 2017. **Why It Matters** - **Universal**: Can be inserted into any CNN (ResNet-SE, MobileNet-SE, EfficientNet uses SE). - **Minimal Cost**: Only ~1% more parameters and ~0.5% more FLOPs for 1-2% accuracy improvement. - **Attention Pioneer**: One of the first channel attention mechanisms, inspiring CBAM, ECA, and GE-Net. **SE** is **the channel importance learner** — teaching the network to amplify useful channels and suppress uninformative ones.