autoregressive diffusion

**Autoregressive Diffusion** is a **hybrid generative model that combines autoregressive (left-to-right) generation with diffusion-based denoising** — generating tokens sequentially but using a diffusion process at each position, or applying diffusion with an autoregressive ordering constraint. **Autoregressive Diffusion Variants** - **ARDM (Autoregressive Diffusion Models)**: Generate tokens in a random order — each token is generated conditioned on previously generated tokens. - **Order-Agnostic**: Learn to generate in ANY order, not just left-to-right — order is sampled during training. - **Upsampling**: Generate a coarse sequence autoregressively, then refine with diffusion — hierarchical approach. - **Absorbing + AR**: Combine absorbing diffusion (unmask one token at a time) with autoregressive conditioning. **Why It Matters** - **Flexibility**: Unlike pure AR models (fixed left-to-right), ARDM can generate in any order — more flexible decoding. - **Quality**: Combining AR conditioning with diffusion can improve generation quality over pure non-autoregressive methods. - **Speed**: Can decode faster than pure AR (generate multiple tokens per step) while maintaining coherence. **Autoregressive Diffusion** is **sequential denoising** — combining the coherence of autoregressive generation with the flexibility and quality of diffusion models.