multimodal chain-of-thought

**Multimodal Chain-of-Thought** is a **prompting strategy that encourages models to reason across modalities step-by-step** — fusing visual evidence with textual knowledge to solve problems that neither modality could solve alone. **What Is Multimodal CoT?** - **Definition**: Scaffolding reasoning using both text and image intermediates. - **Example**: "What is unusual about this image?" - **Step 1 (Vision)**: "I see a man ironing clothes." - **Step 2 (Vision)**: "I see he is ironing on the back of a taxi." - **Step 3 (Knowledge)**: "Ironing is usually done indoors on a board." - **Conclusion**: "This is an example of 'extreme ironing', a humor sport." **Why It Matters** - **Synergy**: Text provides the world knowledge (physics, culture); Vision provides the facts. - **Complex QA**: Necessary for ScienceQA (interpreting diagrams + formulas). - **Reduced Hallucinatons**: Grounding each step prevents the model from drifting into fantasy. **Multimodal Chain-of-Thought** is **the synthesis of perception and cognition** — allowing AI to apply textbook knowledge to real-world visual observations.