ConvMixer is a minimalist vision architecture that uses only standard depthwise separable convolutions for both spatial mixing and channel mixing, demonstrating that the "patching" strategy (dividing images into non-overlapping patches) introduced by Vision Transformers—not the attention mechanism—is a key ingredient for strong performance. ConvMixer applies a large-kernel depthwise convolution for spatial mixing and a pointwise (1×1) convolution for channel mixing, achieving competitive accuracy with extreme architectural simplicity.

Why ConvMixer Matters in AI/ML: ConvMixer demonstrated that patch embedding is the critical innovation from ViTs, not self-attention, and that even simple convolutional architectures can match ViT performance when they adopt the same patch-based input processing strategy.

• Patch embedding — Like ViT and MLP-Mixer, ConvMixer first divides the input image into non-overlapping patches using a large-stride convolution (kernel=patch_size, stride=patch_size); this aggressive downsampling is the shared innovation across modern architectures • Depthwise convolution — Spatial mixing uses depthwise convolution with large kernels (7×7 to 9×9): each channel is convolved independently, providing local spatial interaction without mixing channel information; this replaces both attention and MLP-based token mixing • Pointwise (1×1) convolution — Channel mixing uses standard 1×1 convolutions that mix information across channels independently per spatial location, equivalent to a per-patch linear layer; this is the simplest possible channel interaction • Isotropic design — Like ViT and MLP-Mixer, ConvMixer uses a uniform resolution throughout the network (no downsampling pyramid), processing patch tokens at constant spatial resolution through all layers • Simplicity as a feature — ConvMixer has only three hyperparameters beyond depth: patch size, hidden dimension, and kernel size; this extreme simplicity makes it an ideal baseline for understanding which architectural components truly matter

ConvMixer is the minimalist proof that the patch embedding strategy—not attention—is the transformative innovation from Vision Transformers, demonstrating that simple depthwise convolutions with aggressive patch-based input processing achieve competitive image classification accuracy with extreme architectural simplicity.