eca

**ECA** (Efficient Channel Attention) is a **lightweight channel attention mechanism that captures local cross-channel interactions using a 1D convolution** — avoiding the dimensionality reduction (FC bottleneck) used in SE-Net, which loses information about direct channel correspondence. **How Does ECA Work?** - **Global Average Pooling**: Squeeze spatial dimensions: $z in mathbb{R}^C$. - **1D Convolution**: Apply a 1D conv of kernel size $k$ on $z$ (captures local channel interactions). - **Adaptive $k$**: $k = |frac{log_2 C}{gamma} + frac{b}{gamma}|_{odd}$ (kernel size adapts to channel count). - **Sigmoid**: Produce per-channel attention weights. - **Paper**: Wang et al. (2020). **Why It Matters** - **No FC Bottleneck**: Avoids the information loss from SE-Net's channel reduction/expansion MLP. - **Fewer Parameters**: One 1D conv layer vs. SE's two FC layers — dramatically fewer parameters. - **Same or Better Accuracy**: Matches or exceeds SE-Net performance with much lower overhead. **ECA** is **SE-Net without the bottleneck** — using a simple 1D convolution to capture channel dependencies efficiently and without information loss.