SimCLR is a contrastive self-supervised framework learning visual representations through data augmentation. Core idea: Different augmentations of same image should have similar embeddings, different images should have different embeddings. Method: Take image → create two augmented views → encode both with same network → project to embedding space → contrastive loss (NT-Xent) maximizes agreement between views of same image. Key components: Strong data augmentations (crop, color, blur), large batch sizes (4096+), projection head (discarded after training), temperature-scaled contrastive loss. Data augmentation combination: Random crop + resize + color distortion + Gaussian blur. Composition crucial for performance. NT-Xent loss: Normalized temperature-scaled cross entropy. Treats one view's positives against all other views as negatives. Representation usage: Discard projection head, use encoder representations for downstream tasks. Fine-tune or linear probe. Results: Competitive with supervised pre-training on ImageNet with enough compute. SimCLR v2: Larger models, MoCo-style memory bank, distillation. Impact: Demonstrated power of contrastive learning, influenced many subsequent methods.
Explore 500+ Semiconductor & AI Topics
From EUV lithography to CUDA optimization — search the full knowledge base or chat with our AI assistant.