Diffusion Language Models

Keywords: diffusion language models, generative models

Diffusion Language Models apply the diffusion-denoising framework to discrete text generation — adapting the successful image diffusion approach to language by handling the challenge of discrete tokens, enabling non-autoregressive generation, iterative refinement, and controllable text generation, an active research area bridging image and language generation paradigms.

What Are Diffusion Language Models?

• Definition: Language models using diffusion process for text generation.
• Challenge: Text is discrete (tokens) while standard diffusion operates on continuous values.
• Goal: Apply diffusion benefits (iterative refinement, controllability) to text.
• Status: Active research, not yet mainstream like autoregressive models.

Why Diffusion for Language?

• Non-Autoregressive: Generate multiple tokens in parallel, not left-to-right.
• Iterative Refinement: Edit and improve text over multiple steps.
• Controllable Generation: Easier to guide generation with constraints.
• Flexible Editing: Modify specific parts while keeping others fixed.
• Theoretical Appeal: Unified framework with image generation.

The Discrete Challenge

Continuous Diffusion (Images):

• Forward: Gradually add Gaussian noise to image.
• Reverse: Learn to denoise, recover original image.
• Works: Images are continuous pixel values.

Discrete Text Problem:

• Tokens: Text is discrete symbols (words, subwords).
• No Natural Noise: Can't add Gaussian noise to discrete tokens.
• Solution Needed: Adapt diffusion to discrete space.

Approaches to Discrete Diffusion

Embed to Continuous Space:

• Method: Embed tokens to continuous vectors, diffuse, project back.
• Forward: x → embedding → add noise → noisy embedding.
• Reverse: Denoise embedding → project to nearest token.
• Examples: D3PM (Discrete Denoising Diffusion), Analog Bits.
• Challenge: Projection back to discrete space is non-differentiable.

Diffusion in Probability Space:

• Method: Diffuse probability distributions over tokens (simplex).
• Forward: Gradually mix token distribution with uniform distribution.
• Reverse: Learn to recover original distribution.
• Benefit: Stays in probability space, no projection needed.
• Challenge: High-dimensional simplex (vocab size).

Score Matching in Discrete Space:

• Method: Adapt score-based models to discrete variables.
• Forward: Define discrete corruption process.
• Reverse: Learn score function for discrete space.
• Benefit: Principled discrete diffusion.
• Challenge: Computational complexity.

Absorbing State Diffusion:

• Method: Tokens gradually transition to special [MASK] token.
• Forward: Replace tokens with [MASK] with increasing probability.
• Reverse: Predict original tokens from masked sequence.
• Connection: Similar to BERT masked language modeling.
• Examples: D3PM, MDLM (Masked Diffusion Language Model).

Training Process

Forward Process (Corruption):

• Step 1: Start with clean text sequence.
• Step 2: Apply corruption (masking, replacement, noise) with schedule.
• Step 3: Generate corrupted sequences at different noise levels.
• Schedule: Typically linear or cosine schedule over T steps.

Reverse Process (Denoising):

• Model: Transformer predicts less-corrupted version from corrupted input.
• Input: Corrupted sequence + noise level (timestep embedding).
• Output: Predicted cleaner sequence or denoising direction.
• Loss: Cross-entropy between predicted and target tokens.

Sampling (Generation):

• Start: Begin with fully corrupted sequence (all [MASK] or random).
• Iterate: Gradually denoise over T steps.
• Step: At each step, predict less noisy version, add controlled noise.
• End: Final sequence is generated text.

Benefits of Diffusion for Language

Non-Autoregressive Generation:

• Parallel: Generate all tokens simultaneously (in principle).
• Speed: Potential for faster generation than autoregressive.
• Reality: Still requires multiple diffusion steps, not always faster.

Iterative Refinement:

• Multiple Passes: Refine text over multiple denoising steps.
• Edit Capability: Modify specific tokens while keeping others.
• Quality: Iterative refinement can improve coherence.

Controllable Generation:

• Guidance: Easier to apply constraints during generation.
• Infilling: Fill in missing parts of text naturally.
• Conditional: Condition on various signals (sentiment, style, content).

Flexible Editing:

• Partial Editing: Modify specific spans, keep rest unchanged.
• Inpainting: Fill in masked regions conditioned on context.
• Rewriting: Iteratively improve specific aspects.

Challenges

Discrete Nature:

• Fundamental: Text discreteness doesn't match continuous diffusion.
• Workarounds: All approaches have trade-offs.
• Performance: Not yet matching autoregressive quality on most tasks.

Computational Cost:

• Multiple Steps: Requires T forward passes (typically T=50-1000).
• Slower: Often slower than single autoregressive pass.
• Trade-Off: Quality vs. speed.

Training Complexity:

• Noise Schedule: Requires careful tuning of corruption schedule.
• Hyperparameters: More hyperparameters than autoregressive.
• Stability: Training can be less stable.

Evaluation:

• Metrics: Standard metrics (perplexity, BLEU) may not capture benefits.
• Quality: Human evaluation needed for iterative refinement quality.

Current State & Research

Active Research Area:

• Many Approaches: D3PM, MDLM, Analog Bits, DiffuSeq, and more.
• Improving: Performance gap with autoregressive narrowing.
• Applications: Exploring where diffusion excels (editing, infilling).

Competitive on Some Tasks:

• Infilling: Better than autoregressive for filling masked spans.
• Controllable Generation: Easier to apply constraints.
• Paraphrasing: Iterative refinement useful for rewriting.

Not Yet Mainstream:

• Autoregressive Dominance: GPT-style models still dominant.
• Scaling: Unclear if diffusion benefits scale to very large models.
• Adoption: Limited production deployment so far.

Applications

Text Infilling:

• Task: Fill in missing parts of text.
• Advantage: Diffusion naturally handles bidirectional context.
• Use Case: Document completion, story writing.

Controlled Generation:

• Task: Generate text with specific attributes (sentiment, style).
• Advantage: Easier to apply guidance during diffusion.
• Use Case: Controllable story generation, style transfer.

Text Editing:

• Task: Modify specific parts of text.
• Advantage: Iterative refinement, partial editing.
• Use Case: Paraphrasing, rewriting, improvement.

Machine Translation:

• Task: Translate between languages.
• Advantage: Non-autoregressive, iterative refinement.
• Use Case: Fast translation with quality refinement.

Tools & Implementations

• Diffusers (Hugging Face): Includes some text diffusion models.
• Research Code: D3PM, MDLM implementations on GitHub.
• Experimental: Not yet in production frameworks like GPT.

Diffusion Language Models are an exciting research frontier — while not yet matching autoregressive models in general text generation, they offer unique advantages in controllability, editing, and infilling, and represent an important exploration of alternative paradigms for language generation that may unlock new capabilities as the field matures.

Source: ChipFoundryServices — Search this topic — Ask CFSGPT