ELECTRA uses replaced token detection instead of masking for more efficient and effective pre-training. Key innovation: Instead of masking and predicting tokens, train model to detect which tokens were replaced by a small generator. Architecture: Generator (small MLM model) proposes replacements, discriminator (main model) identifies replaced tokens. Training signal: Every token provides signal (real or replaced?) vs only 15% masked tokens in BERT. More efficient use of compute. Generator: Small BERT-like model trained with MLM, used only for creating training signal. Discriminator: The actual model being trained, learns rich representations from detection task. Efficiency: Matches RoBERTa performance with 1/4 the compute. Much more sample-efficient. Fine-tuning: Use only discriminator (discard generator), fine-tune like BERT for downstream tasks. Results: Strong performance across GLUE, SQuAD, with less pre-training. Variants: ELECTRA-small, base, large. Impact: Influenced efficient pre-training research. Showed alternatives to MLM can be highly effective.