pointwise convolution

**Pointwise Convolution** is a **1×1 convolution that operates across channels at each spatial position independently** — used to change the number of channels (projection), mix channel information, and add nonlinearity without any spatial interaction. **Properties of Pointwise Convolution** - **Kernel Size**: 1×1 (no spatial extent). - **Operation**: Linear combination of channels at each pixel: $y_j(h,w) = sum_i W_{ji} cdot x_i(h,w)$. - **Parameters**: $C_{in} imes C_{out}$ per layer. - **Equivalent To**: A fully connected layer applied to each spatial position independently. **Why It Matters** - **Channel Mixing**: The primary mechanism for inter-channel communication in depthwise-separable convolutions. - **Projection**: Used to reduce or expand channel dimensions (bottleneck design). - **Ubiquitous**: Used in every MobileNet, EfficientNet, ShuffleNet, and modern lightweight architecture. **Pointwise Convolution** is **the channel mixer** — the 1×1 operation that connects information across feature channels at every spatial position.