polarized self-attention

**Polarized Self-Attention (PSA)** is a **dual-branch attention mechanism that computes channel-only and spatial-only self-attention in parallel** — using slim tensors to maintain high resolution with minimal computational overhead. **How Does PSA Work?** - **Channel Branch**: Collapse spatial dimensions -> compute channel self-attention -> broadcast back. - **Spatial Branch**: Collapse channel dimensions -> compute spatial self-attention -> broadcast back. - **Polarized**: Each branch fully collapses the non-target dimension to a size of 1, creating "polarized" attention tensors. - **Combine**: Element-wise multiplication or addition of both branches. - **Paper**: Liu et al. (2021). **Why It Matters** - **Fine-Grained**: Maintains full spatial resolution and full channel resolution simultaneously. - **Efficient**: The "polarization" (collapsing one dimension to 1) makes attention computation very cheap. - **Dense Prediction**: Designed for pixel-level tasks (segmentation, keypoint detection) where resolution matters. **PSA** is **fully polarized dual attention** — collapsing one dimension completely to compute the other efficiently, maintaining precise spatial and channel information.