axial attention for video

**Axial attention for video** is the **factorized attention method that applies separate attention passes along temporal, height, and width axes** - this decomposition reduces complexity while still enabling broad space-time context exchange. **What Is Axial Attention in Video?** - **Definition**: Attention computed one axis at a time instead of over full flattened spatiotemporal sequence. - **Axis Sequence**: Temporal pass, height pass, and width pass in configurable order. - **Complexity Benefit**: Lower cost than full joint attention at comparable receptive reach. - **Use Cases**: Long clips, high resolution inputs, and memory-constrained training. **Why Axial Attention Matters** - **Scalable Context**: Preserves long-range dependencies with manageable token operations. - **Modular Design**: Axis-specific blocks are easy to tune and analyze. - **Hardware Friendliness**: Smaller attention matrices improve throughput. - **Quality Retention**: Often close to joint-attention accuracy when layered effectively. - **Hybrid Compatibility**: Works well with local windows and multiscale backbones. **Axial Video Pipeline** **Temporal Axis Pass**: - Connect corresponding spatial tokens across frames. - Capture motion and event progression. **Spatial Axis Passes**: - Height and width attention propagate contextual structure within frames. - Build spatial coherence after temporal update. **Residual Integration**: - Residual and normalization layers stabilize multi-pass composition. - Deep stacking increases effective receptive field. **How It Works** **Step 1**: - Reshape token tensor to isolate one axis and run attention for that axis only. **Step 2**: - Repeat for remaining axes, merge outputs with residual paths, and continue through network depth. Axial attention for video is **a practical decomposition that approximates global spatiotemporal reasoning at much lower cost** - it is a strong option for long-form or high-resolution video transformers.