dqn

**DQN** (Deep Q-Network) is the **foundational deep reinforcement learning algorithm that combines Q-learning with deep neural networks** — using a CNN to estimate the action-value function $Q(s,a)$ from raw pixel inputs, stabilized by experience replay and a target network. **DQN Innovations** - **Experience Replay**: Store transitions $(s, a, r, s')$ in a replay buffer — sample random mini-batches for training. - **Target Network**: A slowly-updated copy of the Q-network provides stable targets: $y = r + gamma max_{a'} Q_{target}(s', a')$. - **$epsilon$-Greedy**: Explore with probability $epsilon$, exploit with probability $1-epsilon$. - **Loss**: $L = (y - Q_ heta(s, a))^2$ — minimize the temporal difference error. **Why It Matters** - **Breakthrough**: DQN (Mnih et al., 2015) was the first deep RL to achieve human-level performance on Atari games. - **End-to-End**: Learns directly from raw pixels to actions — no hand-crafted features. - **Foundation**: DQN spawned an entire family of improvements (Double DQN, Dueling DQN, Rainbow). **DQN** is **deep learning meets Q-learning** — the algorithm that launched the deep reinforcement learning revolution.