zero-shot segmentation

**Zero-Shot Segmentation** is the **ability to segment objects that were not seen during training** — allowing models to identify and delineate novel categories based on visual similarity, textual descriptions, or generic objectness properties. **What Is Zero-Shot Segmentation?** - **Definition**: Segmenting classes $C_{test}$ disjoint from training classes $C_{train}$. - **Mechanism**: Usually aligns visual features with semantic word embeddings (like CLIP). - **Goal**: Eliminate the need to collect masks for every possible object in the world. - **Input**: Image + Category Name (e.g., "armadillo") -> Output: Mask of armadillo. **Why This Matters** - **Long Tail**: Standard datasets cover ~80 classes (COCO); real world has millions. - **Cost**: Pixel-level annotation is expensive and slow to acquire. - **Scalability**: New categories can be added simply by changing the text vocabulary. **Approaches** - **Open-Vocabulary Segmentation**: Using CLIP-based text encoders to classify pixel regions. - **Class-Agnostic Segmentation**: Models like SAM that segment "objects" regardless of class. - **Generative Approaches**: Using text-to-image diffusion models to generate masks. **Zero-Shot Segmentation** is **the key to open-world understanding** — breaking the constraints of closed-set fixed-vocabulary computer vision systems.