emergent abilities

Emergent abilities in large language models are capabilities that appear suddenly at certain model scales but are not present in smaller models, suggesting qualitative changes in model behavior beyond simple performance improvements. Examples include multi-step arithmetic reasoning, following complex instructions, few-shot learning of new tasks, and chain-of-thought reasoning. These abilities are not explicitly trained but emerge from scale—they appear unpredictably as models cross certain size thresholds (often 10B-100B parameters). The phenomenon suggests that scale enables fundamentally new computational patterns rather than just incremental improvements. Emergent abilities have been observed in reasoning tasks, code generation, multilingual understanding, and instruction following. The mechanisms underlying emergence are debated—possibilities include learning compositional representations, memorizing more training data patterns, or discovering algorithmic solutions. Some researchers question whether emergence is real or an artifact of evaluation metrics. Emergent abilities motivate continued scaling and raise questions about what other capabilities might appear at larger scales. Understanding emergence is critical for predicting and controlling advanced AI systems.