poolformer
**PoolFormer** is the **MetaFormer style architecture that replaces attention with simple pooling as token mixer while retaining strong residual transformer like skeleton** - it argues that the block framework can be more important than the specific mixing operator.
**What Is PoolFormer?**
- **Definition**: A backbone built from MetaFormer blocks where token mixing is done by local pooling rather than attention.
- **MetaFormer Template**: Norm, token mixer, residual, channel MLP, residual.
- **Simple Mixer**: Average pooling layer with small kernel handles spatial interaction.
- **Goal**: Validate whether expensive attention is necessary for strong performance.
**Why PoolFormer Matters**
- **Architectural Insight**: Demonstrates value of block organization and optimization recipe.
- **Efficiency**: Pooling is cheaper and easier to optimize than attention.
- **Stable Training**: Simple operators reduce numerical complexity and training instability.
- **Deployment Ready**: Pooling kernels are universally supported across accelerators.
- **Research Baseline**: Useful for testing new ideas without heavy attention overhead.
**PoolFormer Block Design**
**Token Mixer**:
- Local average pooling injects neighborhood information.
- No dynamic attention weights are computed.
**Channel MLP**:
- Expands and contracts feature channels for semantic transformation.
- Often uses GELU activation and dropout.
**Residual Structure**:
- Two residual paths preserve gradient flow and depth scalability.
**How It Works**
**Step 1**: Patch embedding creates token map, then pooling mixer applies local spatial aggregation inside MetaFormer block.
**Step 2**: Channel MLP refines features, repeated blocks build hierarchy, and final pooled representation is classified.
**Tools & Platforms**
- **timm**: Reference PoolFormer implementations.
- **PyTorch mobile**: Efficient inference due to common pooling and MLP ops.
- **Benchmark suites**: Good baseline for comparing custom token mixers.
PoolFormer is **a minimal yet strong proof that a well designed block scaffold can unlock performance even with very simple mixing operations** - it is a practical and insightful baseline for efficient vision research.