snail

**SNAIL** (Simple Neural Attentive Learner) is a **meta-learning architecture that uses temporal convolutions and attention to aggregate experience** — processing a sequence of observations and labels (or states and rewards) to make predictions for new inputs, combining the local focus of convolutions with the global access of attention. **SNAIL Architecture** - **Temporal Convolutions**: Causal dilated convolutions capture local temporal patterns in the experience sequence. - **Attention**: Soft attention over all previous experiences — enables global access to any past observation. - **Interleaved**: Alternate convolution and attention blocks — convolutions provide features, attention retrieves relevant memories. - **Sequence**: The entire support set is processed as a sequence — each new query can attend to all past examples. **Why It Matters** - **General**: Works for both supervised few-shot learning and meta-RL — a unified architecture. - **Scalable**: Attention handles variable-length experience — no fixed context window. - **Structure**: Temporal convolutions capture local structure that pure attention might miss. **SNAIL** is **the attention-based meta-learner** — combining temporal convolutions and attention to learn from sequential experience for fast adaptation.