depthwise separable convolution

**Depthwise Separable Convolution** — a factorized convolution that dramatically reduces computation and parameters by splitting a standard convolution into two steps, enabling efficient mobile and edge deployment. **Standard Convolution** - Input: $H \times W \times C_{in}$ → Output: $H \times W \times C_{out}$ - One filter: $K \times K \times C_{in}$ (mixes spatial AND channel info simultaneously) - Cost: $K^2 \times C_{in} \times C_{out} \times H \times W$ **Depthwise Separable (Two Steps)** 1. **Depthwise Conv**: One $K \times K$ filter per input channel (spatial only, no channel mixing). Cost: $K^2 \times C_{in} \times H \times W$ 2. **Pointwise Conv**: $1 \times 1$ convolution to mix channels. Cost: $C_{in} \times C_{out} \times H \times W$ **Savings** - Reduction factor: $\frac{1}{C_{out}} + \frac{1}{K^2}$ - For 3x3 conv with 256 output channels: ~8-9x fewer operations **Key Architectures** - **MobileNetV1/V2/V3**: Google's mobile-optimized CNNs using depthwise separable convolutions - **EfficientNet**: NAS-designed architecture using similar factorization - **Xception**: "Extreme Inception" — replaced all convolutions with depthwise separable **Depthwise separable convolutions** make it possible to run powerful vision models on smartphones and IoT devices in real-time.