token pruning

**Token Pruning** is an **efficiency technique that removes uninformative tokens during inference** — reducing the number of tokens processed by subsequent transformer layers, speeding up inference proportionally to the fraction of tokens removed. **How Does Token Pruning Work?** - **Score**: Compute an importance score for each token (CLS attention, gradient magnitude, learned predictor). - **Prune**: Remove tokens with scores below a threshold (or keep top-$k$). - **Continue**: Remaining tokens are processed by subsequent layers with reduced computation. - **Examples**: DynamicViT, EViT, ToMe (Token Merging), A-ViT. **Why It Matters** - **Inference Speed**: 30-50% token reduction → proportional speedup with minimal accuracy loss (<0.5%). - **Adaptive**: Different images have different amounts of informative content — pruning adapts automatically. - **Deployable**: No architectural changes needed — can be applied to pre-trained models. **Token Pruning** is **throwing away what doesn't matter** — dynamically removing uninformative tokens to accelerate transformer inference.