double dqn

**Double DQN** is an **improvement to DQN that addresses the overestimation bias in Q-learning** — using the online network to select the best action and the target network to evaluate it, decoupling action selection from evaluation to reduce systematic overestimation. **Double DQN Fix** - **DQN Problem**: $y = r + gamma max_{a'} Q_{target}(s', a')$ — the same network both selects and evaluates, causing overestimation. - **Double DQN**: $y = r + gamma Q_{target}(s', argmax_{a'} Q_ heta(s', a'))$ — online network selects, target network evaluates. - **Decoupling**: Separating selection and evaluation eliminates the positive bias. - **Simple**: Just one line of code difference from DQN — use online network for argmax. **Why It Matters** - **Overestimation**: DQN's max operator systematically overestimates Q-values — Double DQN eliminates this. - **Better Performance**: Double DQN consistently improves upon DQN across Atari games. - **No Extra Cost**: Same computational cost as DQN — the target network already exists. **Double DQN** is **the overestimation fix** — decoupling action selection from evaluation for more accurate Q-value estimates.