linear probing

**Linear Probing** is an **evaluation protocol for pre-trained representations where a single linear layer is trained on top of frozen features** — used to measure how linearly separable the learned features are, serving as a standardized benchmark for representation quality. **How Does Linear Probing Work?** - **Freeze**: The entire pre-trained backbone. No gradients flow through it. - **Train**: Only a linear classifier (fully connected layer + softmax) on the frozen features. - **Dataset**: Typically ImageNet-1k (1.28M labeled images, 1000 classes). - **Metric**: Top-1 accuracy. Higher = better representations. **Why It Matters** - **Standardized Benchmark**: The primary way to compare SSL methods (SimCLR, MoCo, DINO, MAE, etc.). - **Measures Separability**: If features are linearly separable, the pre-training learned a meaningful structure. - **Conservation**: No fine-tuning means the result strictly measures the pre-trained features, not the model's ability to adapt. **Linear Probing** is **the straight-line test for representations** — measuring whether pre-trained features organize themselves into linearly separable clusters.