decoder only
**Decoder-Only Transformer** is the **dominant architecture for large language models that processes input sequences left-to-right using causal (autoregressive) masking** — generating tokens one at a time where each token can only attend to previous tokens in the sequence, unifying both "understanding" (processing the input prefix) and "generation" (producing new tokens) in a single model stack, as used by GPT-4, Claude, LLaMA, Gemini, and virtually all modern LLMs.
**What Is a Decoder-Only Transformer?**
- **Definition**: A transformer architecture consisting only of decoder blocks with causal self-attention — each position can attend to itself and all previous positions but not future positions, enforced by masking the upper triangle of the attention matrix with negative infinity before softmax.
- **Autoregressive Generation**: Tokens are generated one at a time, left to right — at each step, the model predicts the probability distribution over the vocabulary for the next token, samples or selects a token, appends it to the sequence, and repeats.
- **Causal Masking**: The attention mask ensures position i can only attend to positions 0 through i — this prevents "cheating" during training (the model can't look at future tokens it's supposed to predict) and enables efficient autoregressive generation at inference time.
- **Unified Architecture**: Unlike encoder-decoder models that separate understanding and generation, decoder-only models handle both in one stack — the input prompt is processed as a prefix (like an encoder), and generation continues from where the prefix ends.
**Why Decoder-Only Dominates**
- **Scaling Laws**: Decoder-only architectures have demonstrated the most predictable scaling behavior — performance improves smoothly and predictably with more parameters, data, and compute, as shown by the Chinchilla scaling laws.
- **Simplicity**: One model architecture, one training objective (next-token prediction), one inference procedure — simpler than encoder-decoder which requires separate encoder and decoder passes.
- **KV Caching**: During generation, the Key and Value matrices for all previous tokens can be cached and reused — only the new token's Q, K, V need to be computed at each step, making generation efficient.
- **Prompting Flexibility**: Input and output are just one continuous sequence — "understanding" tasks (classification, extraction) are handled by prompting, and "generation" tasks (writing, coding) are handled naturally.
**Decoder-Only vs. Encoder-Decoder**
| Aspect | Decoder-Only (GPT) | Encoder-Decoder (T5) |
|--------|-------------------|---------------------|
| Attention | Causal (left-to-right) | Bidirectional (encoder) + Causal (decoder) |
| Input Processing | Unidirectional | Bidirectional (full context) |
| Training Objective | Next-token prediction | Span corruption / seq2seq |
| Generation | Continue from prefix | Decode from encoder output |
| Scaling | Proven to 1T+ parameters | Less explored at extreme scale |
| Inference | KV cache for efficiency | Cross-attention adds complexity |
| Dominant Models | GPT-4, Claude, LLaMA, Gemini | T5, BART, mBART |
**Decoder-only transformers are the architecture powering the current generation of large language models** — using causal masking and autoregressive generation to unify language understanding and generation in a single model that scales predictably to hundreds of billions of parameters, establishing the dominant paradigm for AI systems from chatbots to code generation to reasoning.