decoder-only

Decoder-only architecture uses just the decoder portion with causal attention, dominating modern language model design. **Architecture**: Stack of transformer decoder blocks with causal (unidirectional) self-attention. No encoder, no cross-attention. **How it works**: Each layer attends only to previous positions, enabling autoregressive next-token prediction. **Representative models**: GPT series, LLaMA, Claude, Mistral, most production LLMs. **Training objective**: Next token prediction (causal language modeling) on massive text corpora. **Why decoder-only dominates**: Scales predictably, single training objective, handles generation and understanding, emergent abilities at scale. **For understanding tasks**: Reformulate as generation (classification as generating class name, QA as generating answer). **Advantages**: Simpler architecture, efficient training, excellent generation, in-context learning capability. **Comparison to encoder-only**: Less efficient for pure understanding tasks, but more versatile overall. **Efficiency features**: KV caching, parallel training despite sequential generation. **Current landscape**: OpenAI, Anthropic, Meta, Google all using decoder-only for flagship models.