adaptive inference

**Adaptive Inference** is the **dynamic adjustment of a neural network's computational effort based on the difficulty of each input** — allocating more computation to hard inputs and less to easy inputs, optimizing the average computation per sample while maintaining accuracy. **Adaptive Inference Mechanisms** - **Early Exit**: Skip later layers for confident predictions (BranchyNet, MSDNet). - **Dynamic Depth**: Choose how many layers to execute per input (SkipNet, BlockDrop). - **Dynamic Width**: Choose how many channels/filters to use per input (Slimmable Networks). - **Dynamic Resolution**: Process easy inputs at lower resolution, hard inputs at higher resolution. **Why It Matters** - **Efficiency**: Easy inputs (majority in many applications) require much less computation — 2-10× average speedup. - **Budget-Aware**: Set a computation budget and the network adapts to meet it. - **Semiconductor**: Defect images vary in difficulty — simple good/bad decisions exit early, ambiguous defects get full computation. **Adaptive Inference** is **thinking harder when it matters** — dynamically allocating computation based on each input's difficulty for efficient inference.