mlp-mixer for vision

**MLP-Mixer** is the **canonical all-MLP vision architecture that alternates token-mixing and channel-mixing layers on patch embeddings** - it demonstrates that global spatial interaction can be learned through transposed MLP operations without any attention mechanism. **What Is MLP-Mixer?** - **Definition**: A stack of residual blocks where one MLP mixes information across tokens and a second MLP mixes information across channels. - **Patch Backbone**: Input image is patchified and linearly projected to a fixed embedding dimension. - **Two Mixer Axes**: Token mixing handles spatial relationships, channel mixing handles semantic feature transformation. - **Classifier Head**: Global average pooling plus linear layer predicts class probabilities. **Why MLP-Mixer Matters** - **Conceptual Clarity**: Separates spatial and feature computation into explicit stages. - **Strong Baseline**: Competitive accuracy when trained with large data and modern augmentation. - **Efficient Kernels**: Dominated by matrix multiplies that are easy to optimize. - **Research Utility**: Provides a neutral comparison point versus ViT and ConvNet families. - **Transferability**: Architecture extends to audio and multimodal tokens with minor adaptation. **Block Anatomy** **Token-Mixing MLP**: - Operates on transposed tensor so each channel mixes across all tokens. - Captures long range dependencies across the full image grid. **Channel-Mixing MLP**: - Operates per token across channel dimension. - Expands and contracts features with nonlinearity. **Residual + Norm**: - Pre-norm residual design improves optimization in deeper variants. **How It Works** **Step 1**: Convert image to N patch tokens, each with C channels, then apply token-mixing MLP across N for each channel independently. **Step 2**: Apply channel-mixing MLP across C for each token, stack many blocks, pool token outputs, and classify. **Tools & Platforms** - **timm**: Reference Mixer implementations and pretrained checkpoints. - **JAX and Flax**: Common research stack for large Mixer pretraining. - **TensorRT**: Accelerates inference for matmul heavy backbones. MLP-Mixer is **the foundational all-MLP design that proves spatial reasoning does not strictly require attention or convolution** - its clean decomposition makes it one of the most instructive modern vision baselines.