dynamic convolution

**Dynamic Convolution** is a **convolution where the kernel weights are dynamically generated based on the input** — rather than using fixed, learned weights. The kernel adapts to each input, providing input-dependent feature extraction. **How Does Dynamic Convolution Work?** - **Attention Over Kernels**: Maintain $K$ fixed kernel candidates. Generate attention weights $pi_1, ..., pi_K$ from the input via squeeze-excite. - **Aggregate**: $W_{dynamic} = sum_k pi_k cdot W_k$ (weighted sum of kernel candidates). - **Apply**: Use $W_{dynamic}$ for standard convolution on the current input. - **Paper**: Chen et al. (2020). **Why It Matters** - **Adaptive**: Different inputs get different effective kernels -> more expressive than static kernels. - **Lightweight**: Only adds a small attention module to generate kernel weights. - **MobileNets**: Particularly effective for lightweight models where increasing width is too expensive. **Dynamic Convolution** is **input-adaptive filtering** — generating custom convolutional kernels on the fly for each input.