emergent capability

Emergent capabilities are abilities that appear in large language models at certain scales but are absent or minimal in smaller models, exhibiting phase transitions where performance suddenly improves dramatically rather than gradually scaling with model size. Examples include: chain-of-thought reasoning (multi-step logical deduction), arithmetic and mathematical problem solving, code generation and debugging, multi-lingual translation without parallel training data, and in-context learning from few examples. The emergence phenomenon: plot performance versus model size (parameters, compute, data)—below threshold, near-random performance; above threshold, rapid improvement to high accuracy. This unpredictability challenges scaling laws: smooth loss curves hide capability discontinuities. Hypotheses for emergence: critical mass of relevant knowledge (enough facts to reason), compositional generalization threshold (combining learned skills), and sample complexity (larger models learn more efficiently). Debate: some argue emergence is measurement artifact (different metrics show smoother scaling), while others see genuine capability transitions. Implications: predicting capabilities of future models is difficult, safety considerations become uncertainty-bounded, and emergent risks (deception, manipulation) may appear unexpectedly. Understanding emergence is crucial for AI development planning and governance as models continue scaling.