diffusion model sampling
**Diffusion Model Sampling and Inference** covers the **techniques for generating high-quality samples from trained diffusion models** — including DDPM's stochastic sampling, DDIM's deterministic fast sampling, classifier-free guidance for controllable generation, and advanced schedulers (DPM-Solver, Euler) that reduce the number of denoising steps from 1000 to as few as 1-4 while maintaining quality.
**The Diffusion Process**
```
Forward (noising): x₀ → x₁ → ... → x_T ≈ N(0,I)
q(x_t | x_{t-1}) = N(x_t; √(1-β_t)·x_{t-1}, β_t·I)
Reverse (denoising): x_T → x_{T-1} → ... → x₀ (generated image)
p_θ(x_{t-1} | x_t) = N(x_{t-1}; μ_θ(x_t, t), σ²_t·I)
The neural network predicts ε_θ(x_t, t) — the noise to remove
```
**DDPM (Denoising Diffusion Probabilistic Models)**
Original sampling: iterate T=1000 steps, each adding a small amount of Gaussian noise:
```python
# DDPM sampling (stochastic)
x = torch.randn(shape) # Start from pure noise
for t in reversed(range(T)): # T=1000 steps
predicted_noise = model(x, t)
x = (1/√α_t) * (x - (β_t/√(1-ᾱ_t)) * predicted_noise)
if t > 0:
x += σ_t * torch.randn_like(x) # stochastic noise
```
Slow: 1000 forward passes through the U-Net for one image.
**DDIM (Denoising Diffusion Implicit Models)**
Key insight: derive a **deterministic** sampling process that skips steps:
```python
# DDIM: deterministic, can use S << T steps (e.g., S=50)
for i, t in enumerate(reversed(subsequence)): # S=50 steps
pred_noise = model(x, t)
pred_x0 = (x - √(1-ᾱ_t) * pred_noise) / √ᾱ_t
x = √ᾱ_{t-1} * pred_x0 + √(1-ᾱ_{t-1}) * pred_noise
# No random noise! Deterministic mapping from x_T → x_0
```
Benefits: 20× fewer steps (50 vs 1000), deterministic (same noise → same image), enables interpolation in latent space.
**Classifier-Free Guidance (CFG)**
The most impactful technique for controllable generation:
```python
# During training: randomly drop conditioning c with probability p_drop
# During inference: combine conditional and unconditional predictions
pred_uncond = model(x_t, t, null_condition) # unconditional
pred_cond = model(x_t, t, condition) # conditional (text prompt)
pred = pred_uncond + w * (pred_cond - pred_uncond) # w = guidance scale
# w=1: no guidance, w=7.5: typical for Stable Diffusion, w>10: strong guidance
```
Higher guidance scale → images more closely match the text prompt but with less diversity and potential artifacts. CFG essentially amplifies the signal from the conditioning.
**Advanced Samplers**
| Sampler | Steps | Type | Key Idea |
|---------|-------|------|----------|
| DDPM | 1000 | Stochastic | Original, slow but high quality |
| DDIM | 50-100 | Deterministic | Skip steps, interpolatable |
| DPM-Solver++ | 15-25 | Deterministic | ODE solver, exponential integrator |
| Euler/Euler-a | 20-50 | Both | Simple ODE integration |
| LCM | 2-8 | Deterministic | Consistency distillation |
| SDXL Turbo | 1-4 | Deterministic | Adversarial distillation |
**Noise Schedules**
The sequence of noise levels β₁...β_T significantly affects quality:
- **Linear**: β linearly from 10⁻⁴ to 0.02 (original DDPM)
- **Cosine**: smoother transition, better for small images
- **Scaled linear**: used in Stable Diffusion, shifted for latent space
**Diffusion sampling optimization has been the key enabler of practical generative AI** — reducing generation from minutes (1000-step DDPM) to sub-second (1-4 step distilled models) while maintaining the remarkable quality and controllability that made diffusion models the dominant paradigm for image and video generation.