capsule network

**Capsule Network** is a **neural network architecture that uses groups of neurons (capsules) to encode both the presence and pose of features** — addressing a fundamental limitation of CNNs that discard spatial relationships between features during pooling. **What Is a Capsule?** - **Definition**: A vector of neurons whose length encodes feature probability and direction encodes instantiation parameters (pose, position, scale, orientation). - **vs. Neuron**: A single neuron outputs a scalar; a capsule outputs a vector. - **Key Property**: Capsules preserve spatial hierarchies — where features are relative to each other. **Why Capsule Networks Matter** - **Viewpoint Equivariance**: Capsules recognize objects regardless of orientation — CNNs require extensive augmentation to achieve this. - **Part-Whole Relationships**: A face capsule activates only when eye/nose/mouth capsules agree on consistent pose. - **Fewer Data**: Parse spatial structure more explicitly, potentially learning from fewer examples. - **No Pooling Required**: Dynamic routing replaces pooling, preserving spatial information. **Dynamic Routing Algorithm** - Lower-level capsules send predictions to higher-level capsules. - If predictions agree, routing coefficient increases (iterative agreement). - Runs 3-5 iterations per forward pass. - Computationally expensive — main practical limitation. **Key Papers and Variants** - **CapsNet (Hinton et al., 2017)**: Original capsule architecture, MNIST 99.75% accuracy. - **EM Routing (2018)**: Expectation-maximization instead of dynamic routing. - **Efficient-CapsNet**: Lightweight variant for embedded deployment. **Limitations** - Slow training due to iterative routing. - Doesn't scale well to ImageNet-level tasks (yet). - Harder to implement than standard CNNs. Capsule Networks are **a promising rethinking of how neural networks should represent visual information** — though they have not yet displaced CNNs for large-scale practical applications.