dexperts
**DExperts** is the **decoding-time controllable generation method that combines an expert language model (trained on desired text) with an anti-expert model (trained on undesired text) to steer generation** — developed at the Allen Institute for AI as a simple yet effective approach to controlling attributes like toxicity, sentiment, and formality by ensembling contrasting models during token-level decoding.
**What Is DExperts?**
- **Definition**: A decoding strategy that combines three models at generation time: a base model, an expert model (fine-tuned on desired-attribute text), and an anti-expert model (fine-tuned on undesired-attribute text).
- **Core Innovation**: The expert/anti-expert contrast provides a clean signal for desired attributes, applied at the token probability level during generation.
- **Key Formula**: P(token) = P_base(token) × P_expert(token) / P_anti-expert(token) — amplify expert preferences, suppress anti-expert tendencies.
- **Publication**: Liu et al. (2021), Allen Institute for AI (AI2).
**Why DExperts Matters**
- **Simplicity**: The expert/anti-expert framework is conceptually simple and easy to implement.
- **Effectiveness**: Achieves strong detoxification with minimal fluency degradation — often outperforming more complex methods.
- **No Base Model Changes**: Like GeDi, DExperts works with frozen base models as a decoding-time intervention.
- **Interpretable**: The expert/anti-expert contrast makes the control mechanism transparent and debuggable.
- **Composable**: Multiple attribute controls can be stacked by combining multiple expert/anti-expert pairs.
**How DExperts Works**
**Expert Training**: Fine-tune a small LM on text with the desired attribute (e.g., non-toxic, formal, positive sentiment).
**Anti-Expert Training**: Fine-tune a small LM on text with the undesired attribute (e.g., toxic, informal, negative sentiment).
**Decoding**: At each generation step:
1. Get base model next-token distribution.
2. Get expert model next-token distribution.
3. Get anti-expert model next-token distribution.
4. Combine: multiply base by expert, divide by anti-expert.
5. Sample the next token from the adjusted distribution.
**Performance on Detoxification**
| Method | Toxicity ↓ | Fluency | Diversity |
|--------|-----------|---------|-----------|
| **Base Model** | 0.52 | High | High |
| **PPLM** | 0.32 | Medium | Medium |
| **GeDi** | 0.17 | High | Medium |
| **DExperts** | 0.14 | High | High |
**Advantages Over Alternatives**
- **vs. PPLM**: No gradient computation during generation — much faster inference.
- **vs. Prompting**: Stronger attribute control that doesn't depend on model following instructions.
- **vs. RLHF**: No expensive reinforcement learning training — just two small fine-tuned models.
- **vs. Filtering**: Proactive control during generation rather than reactive rejection of complete outputs.
DExperts is **a clean, effective framework for controlled text generation** — demonstrating that the contrast between expert and anti-expert models provides a powerful, interpretable signal for steering language model outputs toward desired attributes at decoding time.