adaptive token selection

**Adaptive Token Selection** is an **efficiency technique where the model dynamically selects a subset of tokens to process based on the input difficulty** — easier inputs use fewer tokens (and exit earlier), while harder inputs use more computation, creating an input-adaptive compute budget. **How Does Adaptive Token Selection Work?** - **Halting Score**: Each token receives a halting probability at each layer (like Adaptive Computation Time). - **Exit Criterion**: Tokens that have "converged" (halting score > threshold) stop being processed. - **Budget Control**: A regularization loss encourages the model to halt early when possible. - **Example**: A-ViT (Adaptive Vision Transformer) uses per-token adaptive halting. **Why It Matters** - **Input-Dependent Compute**: Simple images (clear sky) use fewer tokens/layers than complex scenes. - **No Fixed Budget**: Unlike static pruning, the compute budget adapts to each input dynamically. - **Anytime Prediction**: Can produce predictions at any computational budget by halting earlier. **Adaptive Token Selection** is **pay-per-difficulty inference** — allocating more computation to hard inputs and saving resources on easy ones.