inverse reinforcement learning

**IRL** (Inverse Reinforcement Learning) is the **problem of recovering the reward function from expert demonstrations** — given an expert's behavior, IRL solves for the reward function that makes the expert's policy optimal, then uses this reward to train a new policy via standard RL. **IRL Methods** - **MaxEnt IRL**: Find the reward function under which the expert's behavior is maximum entropy optimal. - **Feature Matching**: Find a reward such that the learned policy's expected features match the expert's. - **Bayesian IRL**: Posterior over reward functions — captures reward uncertainty. - **Deep IRL**: Parameterize the reward function with a neural network — scales to high-dimensional spaces. **Why It Matters** - **Transferable**: The recovered reward function transfers to new environments — more general than a copied policy. - **Understanding**: The reward reveals WHAT the expert is optimizing — interpretable understanding of expert behavior. - **Ambiguity**: Many reward functions can explain the same behavior — IRL is inherently ill-posed. **IRL** is **inferring WHY the expert acts** — recovering the hidden reward function from observed expert behavior.