meta-learning view of icl

**Meta-learning view of ICL** is the **perspective that language models perform implicit learning algorithms at inference time using prompt examples as training data** - it treats forward-pass adaptation as learned optimization behavior acquired during pretraining. **What Is Meta-learning view of ICL?** - **Definition**: Model is interpreted as implementing task adaptation rules encoded in parameters. - **Inference Learning**: Prompt demonstrations act like mini-training episodes processed at runtime. - **Behavior Signature**: ICL improves as demonstrations become more representative and structured. - **Relation**: Complementary to Bayesian views, with focus on learned update dynamics. **Why Meta-learning view of ICL Matters** - **Capability Explanation**: Helps explain why larger models show stronger few-shot adaptation. - **Prompt Strategy**: Suggests examples should expose task function clearly and consistently. - **Architecture Insight**: Motivates analysis of circuits that implement in-forward adaptation. - **Benchmarking**: Frames ICL tasks as tests of learned meta-optimization ability. - **Safety**: Adaptive behavior can generalize both helpful and harmful patterns quickly. **How It Is Used in Practice** - **Episode Design**: Construct prompts as clean support-set and query-set structures. - **Scaling Analysis**: Compare meta-learning signatures across model sizes and checkpoints. - **Circuit Mapping**: Use patching to identify components that mediate runtime adaptation. Meta-learning view of ICL is **a dynamic-learning interpretation of prompt-based model adaptation** - meta-learning view of ICL is most useful when linked to measurable adaptation dynamics and causal mechanisms.