Few-Shot Examples
Few-shot examples are a handful of worked input-and-output demonstrations placed in a prompt so that a language model infers the desired task, format and style from them, without any change to its weights. A prompt that lists three review texts with their sentiment labels and then ends with a fourth review and a blank for the label is a few-shot prompt: the model completes the pattern, performing the task purely from what appears in its context window.
The idea was established by Brown et al. in 2020 in the paper Language Models are Few-Shot Learners, which presented the 175-billion-parameter GPT-3. The authors compared zero-shot (instruction only), one-shot (a single demonstration) and few-shot settings across dozens of tasks and reported that performance often improved as demonstrations were added and as the model grew larger, with no gradient updates at all. They called this in-context learning, and it changed how models are used: instead of fine-tuning a separate model per task, a developer writes examples into a prompt. Few-shot demonstrations also underpin chain-of-thought prompting, where Wei et al. in 2022 showed that exemplars containing step-by-step reasoning make large models reason in steps themselves.
Which examples are chosen matters, and similarity-based selection often beats a fixed set. Liu et al. in 2021, in the paper asking what makes good in-context examples for GPT-3, found that retrieving demonstrations whose embeddings are nearest to the test input generally outperforms random choice, which is why many applications keep a library of labelled cases and retrieve a few per request. Coverage matters as well: examples should span the label classes and the awkward edge cases, include the real input format, and reflect the true distribution, since the model will imitate whatever the examples show, including their length, tone and mistakes. A single unusual example can pull outputs toward it, and examples that contradict the written instruction create confusing mixed signals.
Order sensitivity is a well-documented weakness. Zhao et al. in 2021, in Calibrate Before Use, showed that few-shot accuracy can swing widely with prompt format, example choice and example order, driven by biases such as favouring the most common label in the examples, favouring the most recent label and favouring tokens frequent in the pretraining data. Lu et al. in 2022, in Fantastically Ordered Prompts and Where to Find Them, found that some permutations of the same examples perform near the state of the art while others perform near chance. Larger and instruction-tuned models are less brittle but not immune, so a prompt should be tested under several orderings and balanced label mixes rather than tuned on one lucky arrangement.
What the demonstrations actually teach is subtler than it looks. Min et al. in 2022, in Rethinking the Role of Demonstrations, reported that replacing the correct labels in examples with random ones hurt performance only modestly on many classification tasks, suggesting that demonstrations largely convey the label space, the input distribution and the output format, while the underlying task skill comes from pretraining. This does not make correctness unimportant on tasks with exact answers, and later work shows the picture varies with model size and task. The practical lesson is to treat examples primarily as a specification of format and style.
Trade-offs are cost and context. Every example consumes tokens on every call, so long demonstrations raise latency and spend; with large context windows, many-shot prompting with dozens or hundreds of examples is now possible and can approach fine-tuning on some tasks. When a task needs thousands of examples, strict consistency or a lower per-call cost, fine-tuning may be the better route. Leaked or test-like examples in the prompt can also inflate evaluation scores, so evaluation cases should be kept separate from demonstrations.
Read few-shot examples as teaching by showing: a few carefully chosen, ordered and balanced demonstrations tell a pretrained model what the task looks like, and their quality and arrangement can matter as much as their number.