vision-language pre-training objectives

**Vision-Language Pre-training Objectives** are the **loss functions used to train foundation models on massive unlabelled data** — teaching them to understand the relationship between visual and textual information without explicit human supervision. **Key Objectives** - **ITC (Image-Text Contrastive)**: Global alignment (CLIP style). Maximizes similarity of correct pairs in a batch. - **ITM (Image-Text Matching)**: Binary classification. "Does this text match this image?" using a fusion encoder. - **MLM (Masked Language Modeling)**: BERT-style. Predict missing words in a caption given the image context. - **MIM (Masked Image Modeling)**: Predict missing image patches given the text. - **LM (Language Modeling)**: Autoregressive generation (GPT style). "Given image, generate caption." **Why They Matter** - **Self-Supervision**: Allows training on billions of noisy web pairs (LAION-5B) rather than thousands of labeled datasets. - **Robustness**: The combination of objectives (e.g., ITC + ITM + LM in BLIP) produces the strongest features. **Vision-Language Pre-training Objectives** are **the curriculum for AI education** — defining exactly what the model "studies" to become intelligent.