triplet attention

**Triplet Attention** is a **lightweight attention mechanism that computes cross-dimension interactions between channel and spatial dimensions** — using three parallel branches to capture (C×H), (C×W), and (H×W) attention, without any dimensionality reduction. **How Does Triplet Attention Work?** - **Branch 1**: Rotate tensor to (H, C, W) -> compute attention on (C, W) plane. - **Branch 2**: Rotate tensor to (W, H, C) -> compute attention on (H, C) plane. - **Branch 3**: Standard spatial attention on (H, W) plane. - **Aggregate**: Average the outputs of all three branches. - **Paper**: Misra et al. (2021). **Why It Matters** - **No Reduction**: Unlike SE/CBAM, uses no dimensionality reduction (MLP bottleneck) -> preserves all information. - **Cross-Dimension**: Captures interactions between channel and spatial dimensions that separate attention misses. - **Negligible Cost**: Almost zero additional parameters (only uses 7×7 convolutions for attention). **Triplet Attention** is **three-way cross-dimensional attention** — capturing every possible interaction between channel, height, and width dimensions.