RoBERTa is a robustly optimized BERT that improved pre-training to achieve better performance without architecture changes. Key improvements over BERT: Longer training: 10x more data, more steps. Larger batches: 8K batch size vs 256. No NSP: Removed Next Sentence Prediction (found harmful). Dynamic masking: Different mask each epoch vs static. More data: BookCorpus + CC-News + OpenWebText + Stories. Results: Significant gains on all benchmarks over BERT with same architecture. Proved BERT was undertrained. Architecture: Identical to BERT - just better training recipe. Variants: RoBERTa-base, RoBERTa-large matching BERT sizes. Impact: Showed importance of training decisions, influenced subsequent models. Use cases: Same as BERT - classification, NER, embeddings, extractive QA. Often preferred over BERT due to better performance. Tokenizer: Uses byte-level BPE (like GPT-2) instead of WordPiece. Legacy: Demonstrated that training recipe matters as much as architecture innovation.