VCR (Visual Commonsense Reasoning) is a benchmark that tests "Theory of Mind" for AI — requiring models not just to answer questions about an image, but to provide the rationale for why that answer is correct, often involving social cues and unstated physical rules.

What Is VCR?

• Definition: A Q&A > R (Question -> Answer -> Rationale) task.
• Structure:

1. Question: "Why is person [1] pointing at person [2]?" 2. Answer: "He is accusing him of stealing." 3. Rationale: "Because person [2] is holding the object behind his back."

• Focus: Social situations, causality, temporal prediction.

Why VCR Matters

• Beyond Recognition: Understanding a scene requires knowing intent, not just pixel labels.
• Safety: Essential for human-robot interaction (understanding if a human is angry, hurried, or joking).
• Cognition: Bridges the gap between Computer Vision and Cognitive Science.

VCR is the empathy test for machines — pushing AI to understand the invisible "why" behind the visible "what".