IQN (Implicit Quantile Network) is a distributional RL algorithm that can sample any quantile of the return distribution — instead of learning a fixed set of quantiles (like QR-DQN), IQN takes a quantile level $ au in [0,1]$ as input and outputs the corresponding quantile value.

IQN Architecture

• Input: State $s$ + sampled quantile level $ au sim U(0,1)$.
• Quantile Embedding: Embed $ au$ using cosine features: $phi( au)_j = ext{ReLU}(sum_i cos(pi i au) w_{ij})$.
• Combination: Hadamard product of state features and quantile embedding.
• Output: The return value at quantile $ au$ for each action — $Z_ au(s,a)$.

Why It Matters

• Arbitrary Quantiles: Can evaluate any quantile at inference — not limited to pre-defined quantile levels.
• Risk Policies: Optimize for any risk level — CVaR, worst-case, or custom risk measures.
• State-of-Art: IQN outperforms both C51 and QR-DQN on Atari benchmarks.

IQN is the universal quantile machine — computing any quantile of the return distribution on-demand for flexible risk-sensitive RL.