vision-language planning

**Vision-Language Planning** is the **ability of an AI to generate a sequence of actionable steps to achieve a goal** — grounding high-level natural language instructions ("Make breakfast") into low-level visual perception and motor capabilities. **What Is Vision-Language Planning?** - **Definition**: Translating "Goal" -> "Plan" using visual context. - **Pipeline**: 1. **Instruction**: "Put the cold apple in the bowl." 2. **Visual Grounding**: Find apple, find fridge (cold), find bowl. 3. **Decomposition**: Open fridge -> Pick apple -> Close fridge -> Find bowl -> Place apple. - **Models**: PaLM-E, RT-2 (Robotic Transformer), SayCan. **Why It Matters** - **Affordance**: The model must understand what is *possible* (can't pick up the table). - **Robotics**: The brain of modern autonomous robots. - **Long-Horizon**: Requires maintaining state over minutes of activity. **Vision-Language Planning** is **the operating system for autonomy** — bridging the gap between abstract human intent and concrete physical actions.