squeeze excite

Squeeze-and-Excitation (SE) blocks add channel attention to convolutional networks by explicitly modeling interdependencies between channels, recalibrating feature maps to emphasize informative channels and suppress less useful ones. The SE block has two operations: squeeze aggregates spatial information into channel descriptors using global average pooling, and excitation learns channel-wise weights through a small fully-connected network with sigmoid activation. These weights scale the original feature maps, adaptively recalibrating channel responses. SE blocks add minimal computational overhead (typically <1% FLOPs) while providing consistent accuracy improvements (1-2% on ImageNet). The mechanism enables the network to perform feature recalibration, emphasizing channels that are most relevant for the current input. SE blocks can be inserted into any CNN architecture (ResNet, Inception, MobileNet) with minimal modification. The approach won the ILSVRC 2017 classification competition. SE blocks represent an early and influential form of attention mechanism in computer vision, predating the widespread adoption of self-attention and vision transformers.