never give up

**NGU** (Never Give Up) is an **exploration algorithm that combines episodic novelty with life-long novelty for persistent exploration** — using both a within-episode novelty signal (encourage visiting new states within the current episode) and a between-episode signal (encourage visiting states not seen in previous episodes). **NGU Components** - **Episodic Novelty**: K-nearest neighbor in an episodic memory of embeddings — reward decreases as similar states accumulate within the episode. - **Life-Long Novelty**: RND-based — detects states novel across all episodes. - **Combined**: $r_i = r_{episodic} cdot min(max(r_{lifelong}, 1), L)$ — multiplicative combination. - **Multiple Policies**: Train a family of policies with different exploration-exploitation trade-offs. **Why It Matters** - **Persistent Exploration**: Unlike pure curiosity (which fades), NGU's episodic component ensures continued exploration. - **State-of-Art**: NGU set new records on hard-exploration Atari games (Montezuma's Revenge, Pitfall). - **Multi-Scale**: Captures novelty at both short-term (episode) and long-term (lifetime) scales. **NGU** is **curiosity that never fades** — combining episodic and life-long novelty for relentless, multi-scale exploration.