image-text contrastive learning

**Image-Text Contrastive Learning (ITC)** is the **dominant pre-training paradigm for aligning vision and language** — training dual encoders to identifying the correct image-text pair from a large batch of random pairings by maximizing the cosine similarity of true pairs. **What Is ITC?** - **Definition**: The "CLIP Loss". - **Mechanism**: 1. Encode $N$ images and $N$ texts. 2. Compute $N imes N$ similarity matrix. 3. Maximize diagonal (correct pairs), minimize off-diagonal (incorrect pairings). - **Scale**: Needs massive batch sizes (e.g., 32,768) to be effective. **Why It Matters** - **Speed**: Decouples vision and text processing, making inference extremely fast (pre-compute embeddings). - **Zero-Shot**: Enables classification without training (just match image to "A photo of a [class]"). - **Robustness**: Learns robust features that transfer to almost any vision task. **Image-Text Contrastive Learning** is **the engine of modern multimodal AI** — providing the foundational embeddings that power everything from image search to generative art.