selective kernel networks

**Selective Kernel (SK) Networks** are a **dynamic kernel selection mechanism that adaptively chooses different convolutional kernel sizes for different inputs** — using an attention mechanism to softly combine features from multiple kernel sizes based on the input content. **How Do SK Networks Work?** - **Split**: Apply convolutions with different kernel sizes (e.g., 3×3 and 5×5) to the same input. - **Fuse**: Add the outputs element-wise -> global average pooling -> compact feature vector. - **Select**: Softmax attention over the kernel branches: $a_k = ext{softmax}(W_k z)$ for each kernel $k$. - **Aggregate**: Final output = weighted sum of branches: $y = sum_k a_k otimes F_k$. - **Paper**: Li et al. (2019). **Why It Matters** - **Adaptive Receptive Field**: The network learns to use small kernels for fine details and large kernels for global context, per-input. - **Content-Dependent**: Different images (or different regions) get different effective kernel sizes. - **Influence**: The dynamic kernel concept influenced subsequent works like CondConv and Dynamic Convolution. **SK Networks** are **neural networks that choose their own kernel size** — dynamically adjusting the receptive field based on what the input needs.