goal-conditioned rl

**Goal-Conditioned RL** is a **reinforcement learning framework where the policy takes both a state and a goal as input** — $pi(a|s,g)$ learns to reach any specified goal $g$, enabling a single policy to accomplish many different tasks by conditioning on different goals. **Goal-Conditioned Components** - **Universal Policy**: $pi(a|s,g)$ — one policy handles all goals by conditioning on the goal. - **Goal Space**: Goals can be target states, images, language descriptions, or abstract representations. - **Reward**: Typically sparse — $r = -mathbf{1}[|s - g| > epsilon]$ — reward only when the goal is reached. - **HER**: Hindsight Experience Replay is essential — relabel failed trajectories with achieved goals. **Why It Matters** - **Generalization**: One policy covers an entire space of goals — no need to retrain for each task. - **Composability**: Goals can be composed sequentially for complex, multi-step tasks. - **Robotics**: Goal-conditioned policies enable flexible robotic manipulation — reach any target position. **Goal-Conditioned RL** is **one policy, any goal** — training a single universal policy to reach any specified goal through conditioning.