Vision-Language Planning is the ability of an AI to generate a sequence of actionable steps to achieve a goal — grounding high-level natural language instructions ("Make breakfast") into low-level visual perception and motor capabilities.

What Is Vision-Language Planning?

• Definition: Translating "Goal" -> "Plan" using visual context.
• Pipeline:

1. Instruction: "Put the cold apple in the bowl." 2. Visual Grounding: Find apple, find fridge (cold), find bowl. 3. Decomposition: Open fridge -> Pick apple -> Close fridge -> Find bowl -> Place apple.

• Models: PaLM-E, RT-2 (Robotic Transformer), SayCan.

Why It Matters

• Affordance: The model must understand what is possible (can't pick up the table).
• Robotics: The brain of modern autonomous robots.
• Long-Horizon: Requires maintaining state over minutes of activity.

Vision-Language Planning is the operating system for autonomy — bridging the gap between abstract human intent and concrete physical actions.