flan-t5

FLAN-T5 is Google's instruction-tuned version of the T5 model, fine-tuned on a massive collection of diverse tasks described via natural language instructions, dramatically improving T5's ability to follow instructions and perform new tasks zero-shot without task-specific examples. FLAN (Fine-tuned LAnguage Net) refers to the instruction tuning methodology, and applying it to T5 produces FLAN-T5 — a model that combines T5's strong text-to-text capabilities with robust instruction following. The FLAN instruction tuning methodology (from "Scaling Instruction-Finetuned Language Models" by Chung et al., 2022) involves fine-tuning on 1,836 tasks grouped into task clusters, with each task expressed through multiple instruction templates — natural language descriptions of what the model should do, such as "Translate the following sentence to French:" or "Is the following movie review positive or negative?" Key advantages of FLAN-T5 over vanilla T5 include: dramatically improved zero-shot performance (following new instructions the model hasn't seen during fine-tuning), improved few-shot performance (better utilizing in-context examples), chain-of-thought reasoning capability (when prompted with "Let's think step by step"), and better instruction following across diverse task formats. FLAN-T5 is available in all T5 sizes: Small (80M), Base (250M), Large (780M), XL (3B), and XXL (11B), making it accessible across hardware configurations. Even FLAN-T5-XL (3B parameters) can outperform much larger models on instruction-following tasks, demonstrating that instruction tuning can be more compute-efficient than pure scaling. FLAN-T5 has become extremely popular in the open-source community for: building task-specific models through further fine-tuning (instruction tuning provides a better starting point than vanilla T5), research experimentation (well-documented, reproducible, and available in multiple sizes), and production deployment (smaller variants run efficiently on modest hardware). FLAN-T5 demonstrated that instruction tuning is a general technique that improves any base model, influencing the development of instruction-tuned variants across the model ecosystem.