t5 (text-to-text transfer transformer)

T5 (Text-to-Text Transfer Transformer) is Google's unified NLP model that reframes every language task as a text-to-text problem — both input and output are always text strings — enabling a single model architecture and training procedure to handle translation, summarization, classification, question answering, and any other NLP task. Introduced by Raffel et al. in the 2020 paper "Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer," T5 demonstrated that this unified framing, combined with large-scale pre-training, achieves state-of-the-art results across diverse benchmarks. The text-to-text framework works by prepending task-specific prefixes to inputs: "translate English to German: [text]," "summarize: [text]," "question: [question] context: [passage]," "classify sentiment: [text]." The model generates the answer as text — "positive" for sentiment, "Berlin" for a factual question, or a full paragraph for summarization. T5 uses the full encoder-decoder transformer architecture (unlike BERT which uses only the encoder, or GPT which uses only the decoder), making it naturally suited for sequence-to-sequence tasks. Pre-training uses a span corruption objective: random contiguous spans of tokens are replaced with sentinel tokens, and the model learns to generate the missing spans — similar to BERT's masking but for multi-token spans. T5 was pre-trained on C4 (Colossal Clean Crawled Corpus — ~750GB of cleaned English web text) in sizes from T5-Small (60M parameters) to T5-11B (11 billion parameters). The paper systematically studied pre-training objectives, architectures, datasets, transfer approaches, and scaling, producing a comprehensive guide to transfer learning best practices. T5's variants include mT5 (multilingual), Flan-T5 (instruction-tuned for improved zero-shot performance), LongT5 (extended context), and UL2 (unified pre-training combining multiple objectives).