blip (bootstrapping language-image pre-training)

**BLIP** (Bootstrapping Language-Image Pre-training) is a **framework for unified vision-language understanding and generation** — which significantly improved performance by cleaning noisy web data using a "Captioner" and "Filter" bootstrapping cycle. **What Is BLIP?** - **Definition**: A VLM pre-training framework. - **Problem Solved**: web image-text pairs are noisy (e.g., filenames as captions). - **Solution**: "CapFilt" (Captioning and Filtering) to generate synthetic captions and filter bad ones. - **Architecture**: Multimodal Mixture of Encoder-Decoder (MED). **Why BLIP Matters** - **Data Quality**: Proved that *clean* synthetic data beats *noisy* real data. - **Versatility**: State-of-the-art on both understanding (VQA, Retrieval) and generation (Captioning). - **Open Source**: The Salesforce implementation became a workhorse model for the community. **Key Components** - **Image-Text Contrastive Loss (ITC)**: Aligns features. - **Image-Text Matching (ITM)**: Binary classification (match/no-match). - **Language Modeling (LM)**: Generates text given image. **BLIP** is **a masterclass in data-centric AI** — demonstrating that how you curate your data is just as important as the model architecture itself.