iqn

**IQN** (Implicit Quantile Network) is a **distributional RL algorithm that can sample any quantile of the return distribution** — instead of learning a fixed set of quantiles (like QR-DQN), IQN takes a quantile level $ au in [0,1]$ as input and outputs the corresponding quantile value. **IQN Architecture** - **Input**: State $s$ + sampled quantile level $ au sim U(0,1)$. - **Quantile Embedding**: Embed $ au$ using cosine features: $phi( au)_j = ext{ReLU}(sum_i cos(pi i au) w_{ij})$. - **Combination**: Hadamard product of state features and quantile embedding. - **Output**: The return value at quantile $ au$ for each action — $Z_ au(s,a)$. **Why It Matters** - **Arbitrary Quantiles**: Can evaluate any quantile at inference — not limited to pre-defined quantile levels. - **Risk Policies**: Optimize for any risk level — CVaR, worst-case, or custom risk measures. - **State-of-Art**: IQN outperforms both C51 and QR-DQN on Atari benchmarks. **IQN** is **the universal quantile machine** — computing any quantile of the return distribution on-demand for flexible risk-sensitive RL.