visual prompting

**Visual Prompting** is an **interaction technique for computer vision models** — where users provide visual cues (points, boxes, scribbles, or reference images) as inputs to guide the model's prediction, rather than relying solely on text or fixed classes. **What Is Visual Prompting?** - **Definition**: Using visual signals to specify the *target* or *task*. - **Examples**: - **Spatial**: Drawing a box around a car to track it. - **Example-based**: Showing an image of a screw and asking "Find all of these". - **Inpainting**: Masking an area to say "fill this space". **Why Visual Prompting Matters** - **Precision**: Text ("the red car") is ambiguous; a click on the pixel is precise. - **New Tasks**: Can define tasks that are hard to describe in words (e.g., "count cells that look abnormal like this one"). - **CV-Native**: Aligns the input modality (visual) with the task modality (visual). **Models**: - **SAM**: Accepts points/boxes. - **SEEM (Segment Everything Everywhere All at Once)**: Accepts audio, visual, and text prompts. - **Visual Prompting (VP)**: Learning pixel-level perturbations to adapt frozen models to new tasks. **Visual Prompting** is **the mouse-click of the AI era** — allowing intuitive, non-verbal communication with intelligent visual systems.