dynamic depth networks

**Dynamic Depth Networks** are **neural networks that adaptively choose how many layers to execute for each input** — skipping unnecessary layers for easy inputs to save computation, while using the full depth for challenging inputs that require more processing. **Dynamic Depth Mechanisms** - **Early Exit**: Attach classifiers at intermediate layers — exit when confident (BranchyNet, MSDNet). - **SkipNet**: Learn a binary gate per residual block — decide to execute or skip each block. - **BlockDrop**: Train a policy to select which blocks to execute, targeting a computation budget. - **Layer Dropping**: Stochastically drop layers during training (regularization), prune at inference. **Why It Matters** - **Computation Savings**: Skipping 30-50% of layers saves proportional computation with <1% accuracy loss. - **Latency Prediction**: The number of executed layers directly determines inference latency. - **Heterogeneous Deploy**: The same model can run at different depths for different hardware budgets. **Dynamic Depth** is **thinking only as deep as needed** — adaptively choosing the number of processing layers based on each input's complexity.