simclr

SimCLR is a contrastive self-supervised framework learning visual representations through data augmentation. **Core idea**: Different augmentations of same image should have similar embeddings, different images should have different embeddings. **Method**: Take image → create two augmented views → encode both with same network → project to embedding space → contrastive loss (NT-Xent) maximizes agreement between views of same image. **Key components**: Strong data augmentations (crop, color, blur), large batch sizes (4096+), projection head (discarded after training), temperature-scaled contrastive loss. **Data augmentation combination**: Random crop + resize + color distortion + Gaussian blur. Composition crucial for performance. **NT-Xent loss**: Normalized temperature-scaled cross entropy. Treats one view's positives against all other views as negatives. **Representation usage**: Discard projection head, use encoder representations for downstream tasks. Fine-tune or linear probe. **Results**: Competitive with supervised pre-training on ImageNet with enough compute. **SimCLR v2**: Larger models, MoCo-style memory bank, distillation. **Impact**: Demonstrated power of contrastive learning, influenced many subsequent methods.