encoder-only

Encoder-only architecture uses just the encoder portion of the transformer, designed for understanding tasks not generation. **Architecture**: Stack of transformer encoder blocks with bidirectional self-attention. No decoder, no cross-attention. **Representative model**: BERT - Bidirectional Encoder Representations from Transformers. **Training objective**: Usually MLM (Masked Language Modeling) - predict masked tokens using bidirectional context. **Output**: Contextualized embeddings for each input token. CLS token embedding often used for classification. **Use cases**: Text classification, named entity recognition, extractive QA, semantic similarity, sentence embeddings. **Why not generation**: Bidirectional attention means no natural left-to-right generation capability. **Fine-tuning**: Add task-specific head (classifier, token labeler) on top of encoder outputs. **Advantages**: Rich bidirectional representations, efficient for understanding tasks, well-suited for embedding extraction. **Models**: BERT, RoBERTa, ELECTRA, ALBERT, DistilBERT. **Current status**: Largely superseded by decoder-only LLMs for many tasks, but still valuable for embeddings and classification.