SAM (Segment Anything Model) is a promptable image segmentation foundation model — capable of cutting out any object in any image based on points, boxes, masks, or text prompts, with zero-shot generalization to unfamiliar objects.

What Is SAM?

• Definition: The first true foundation model for image segmentation.
• Core Capability: "Segment Anything" task — valid mask output for any prompt.
• Dataset: Trained on SA-1B (11 million images, 1.1 billion masks).
• Architecture: Heavy image encoder (ViT) + lightweight prompt encoder + mask decoder.

Why SAM Matters

• Zero-Shot Transfer: Works on underwater, microscopic, or space images without retraining.
• Interactivity: Runs in real-time in the browser (after image embedding computing).
• Ambiguity Handling: Can output multiple valid masks for a single ambiguous point.
• Data Engine: The model-in-the-loop was used to annotate its own training dataset.

How It Works

1. Image Encoder: ViT processes image once to creating an embedding. 2. Prompt Encoder: Processes clicks, boxes, or text into embedding vectors. 3. Mask Decoder: Lightweight transformer combines image and prompt embeddings to predict masks.

SAM is the "GPT" of image segmentation — transforming segmentation from a specialized training task into a generic, promptable capability available to everyone.