rainbow dqn

**Rainbow DQN** is the **combination of six key improvements to DQN into a single integrated agent** — combining Double DQN, Prioritized Experience Replay, Dueling architecture, multi-step returns, distributional RL (C51), and noisy networks for state-of-the-art discrete action RL. **Rainbow Components** - **Double DQN**: Decoupled action selection and evaluation — reduces overestimation. - **PER**: Priority-based replay — focuses on informative transitions. - **Dueling**: Separate value and advantage streams — efficient state value learning. - **Multi-Step**: $n$-step returns instead of 1-step TD — reduces bias, increases variance. - **C51**: Distributional value estimation — learns the full distribution of returns. - **Noisy Nets**: Parametric noise in weights for exploration — replaces $epsilon$-greedy. **Why It Matters** - **Best of All**: Each component contributes independently — combining them yields synergistic improvements. - **Benchmark**: Rainbow set the standard for discrete-action RL when published (Hessel et al., 2018). - **Ablation**: The ablation study showed each component contributes — all six are important. **Rainbow** is **the greatest hits of DQN improvements** — combining six orthogonal enhancements into one powerful agent.