universal value function approximators

**UVFA** (Universal Value Function Approximators) is a **framework for generalizing value functions across goals** — extending standard value functions $V(s)$ to $V(s, g)$ that estimate the expected return from state $s$ when pursuing goal $g$, enabling a single learned function to evaluate any state-goal pair. **UVFA Architecture** - **Input**: State $s$ and goal $g$ — both encoded and combined as input to the value network. - **Generalization**: The network learns to generalize across goals — can predict values for unseen goals. - **Factored**: State and goal can be processed by separate embeddings before being combined. - **Training**: Train on multiple goals simultaneously — Horde architecture for parallel goal learning. **Why It Matters** - **Multi-Goal**: One value function serves all goals — no need to learn separate value functions for each goal. - **Transfer**: Knowledge transfers across goals — similar goals yield similar value estimates. - **Foundation**: UVFAs are the value-function counterpart to goal-conditioned policies — enabling flexible multi-goal RL. **UVFA** is **one value function for all goals** — generalizing value estimation across the entire goal space for efficient multi-goal learning.