diffusion-lm

**Diffusion-LM** is a **language model that applies continuous diffusion to word embeddings for controllable text generation** — mapping discrete tokens to continuous embedding vectors, applying Gaussian diffusion in embedding space, and rounding back to discrete tokens, enabling plug-and-play controllable generation. **Diffusion-LM Architecture** - **Embedding**: Map discrete tokens to continuous embedding vectors — $e(w) in mathbb{R}^d$. - **Forward Diffusion**: Add Gaussian noise to embedding sequence — gradually corrupt the embeddings. - **Reverse Denoising**: Learn to denoise embeddings — predict clean embeddings from noisy ones. - **Rounding**: Map denoised continuous embeddings back to discrete tokens using nearest-neighbor lookup. **Why It Matters** - **Controllability**: Diffusion enables gradient-based control — guide generation toward desired attributes (topic, sentiment, syntax) via classifier guidance. - **Non-Autoregressive**: Generates all positions simultaneously — enables global planning and coherent generation. - **Flexibility**: Plug-and-play classifiers can control any attribute without retraining the base model. **Diffusion-LM** is **diffusion meets language** — applying continuous diffusion in embedding space for flexible, controllable text generation.