information gain exploration

**Information Gain Exploration** is an **exploration strategy that rewards actions that maximize the information gained about the environment** — the agent seeks states and actions that reduce its uncertainty about the transition dynamics, reward function, or other aspects of the MDP. **Information Gain Formulations** - **Bayesian**: Information gain = reduction in posterior uncertainty over model parameters: $I(a; heta | s, D)$. - **VIME**: Variational Information Maximizing Exploration — reward = KL divergence between prior and posterior dynamics. - **Prediction Gain**: Improvement in world model prediction accuracy after experiencing a transition. - **Empowerment**: Information gain about the relationship between actions and future states. **Why It Matters** - **Principled**: Information gain is a theoretically grounded exploration objective — Bayesian optimal design. - **Efficient**: Targets exploration toward states that are most informative — avoids wasting time on irrelevant novelty. - **Model Learning**: Naturally improves the world model — exploration and model learning are synergistic. **Information Gain Exploration** is **seeking the most informative experiences** — exploring where uncertainty is highest to learn the environment fastest.