Pyramid Vision Transformer (PVT) is a hierarchical vision Transformer that introduces progressive spatial reduction across four stages, generating multi-scale feature maps similar to CNN feature pyramids while using self-attention as the core computation. PVT addresses ViT's two key limitations for dense prediction tasks: the lack of multi-scale features and the quadratic complexity of global attention on high-resolution feature maps.

Why PVT Matters in AI/ML: PVT was one of the first pure Transformer backbones for dense prediction (detection, segmentation), demonstrating that Transformers can replace CNNs as general-purpose visual feature extractors when designed with multi-scale output and efficient attention.

• Progressive spatial reduction — PVT processes features through four stages with spatial dimensions [H/4, H/8, H/16, H/32] and increasing channel dimensions [64, 128, 320, 512], producing a feature pyramid identical in structure to ResNet's C2-C5 stages • Spatial Reduction Attention (SRA) — To handle the large number of tokens at early stages (high resolution), PVT reduces the spatial dimension of keys and values by a factor R before computing attention: K̃ = Reshape(K, R)·W_s, reducing complexity from O(N²) to O(N²/R²) • Patch embedding between stages — Overlapping patch embedding layers (strided convolutions) between stages reduce spatial resolution by 2× while increasing channel dimension, serving the same role as pooling/striding in CNNs • Dense prediction compatibility — PVT's multi-scale outputs plug directly into existing detection heads (Feature Pyramid Network, RetinaNet) and segmentation heads (Semantic FPN, UPerNet) designed for CNN feature pyramids • PVTv2 improvements — PVT v2 replaced position embeddings with convolutional position encoding (zero-padding convolution), added overlapping patch embedding, and improved SRA with linear complexity attention, achieving better performance and flexibility

Pyramid Vision Transformer pioneered the hierarchical Transformer backbone for computer vision, demonstrating that multi-scale feature pyramids with spatially reduced attention enable pure Transformer architectures to serve as drop-in replacements for CNN backbones in detection, segmentation, and all dense prediction tasks.