instruction induction
**Instruction Induction** is the **meta-learning technique where a language model infers the underlying task instruction from a set of input-output demonstration examples — automatically generating natural language descriptions of what transformation the examples represent** — the foundational capability that enables automated prompt engineering systems like APE to bootstrap effective instructions without human authoring.
**What Is Instruction Induction?**
- **Definition**: Given a set of (input, output) pairs demonstrating a task, prompting an LLM to describe in natural language what instruction or rule would produce the observed outputs from the given inputs.
- **Meta-Prompt**: "Given these examples, what is the instruction that transforms the inputs into the outputs?" — the model must abstract from specific examples to a general task description.
- **Reverse Engineering Tasks**: The model observes demonstrations of sentiment classification, translation, summarization, or any other task and must articulate what the task is — essentially reverse-engineering the instruction from examples.
- **Foundation for APE**: Instruction induction is the generation step in Automatic Prompt Engineer — producing candidate instructions that are then evaluated and refined.
**Why Instruction Induction Matters**
- **Bootstraps Instructions from Examples**: Many tasks have labeled examples but no written instructions — instruction induction creates the instruction automatically from demonstrations alone.
- **Discovers Effective Phrasings**: The model's generated instructions often use phrasings more aligned with its own training distribution than human-written instructions — leading to better downstream performance.
- **Scalable Task Specification**: Defining hundreds of tasks via examples is faster than writing custom instructions for each — instruction induction automates the conversion from examples to instructions.
- **Meta-Learning Benchmark**: Instruction induction serves as a benchmark for evaluating an LLM's ability to reason about tasks abstractly — measuring whether models understand "what task is being demonstrated."
- **Enables Non-Expert Users**: Users who can provide examples but cannot articulate precise technical instructions benefit from automated instruction generation.
**Instruction Induction Process**
**Phase 1 — Example Presentation**:
- Select 3–10 representative (input, output) pairs from the task dataset.
- Format as clear demonstrations: "Input: [x₁] → Output: [y₁]" for each pair.
- Include diverse examples covering different aspects of the task.
**Phase 2 — Instruction Generation**:
- Prompt the LLM with demonstrations followed by: "What single instruction, when given to a language model along with an input, would produce these outputs?"
- Generate multiple candidate instructions via temperature sampling (N=20–100).
- Candidates range from highly specific to broadly general.
**Phase 3 — Instruction Validation**:
- Test each generated instruction on held-out examples not seen during generation.
- Score by downstream task metric (accuracy, F1, exact match).
- Top-scoring instructions proceed to refinement or deployment.
**Instruction Induction Quality Factors**
| Factor | Impact on Quality | Recommendation |
|--------|------------------|----------------|
| **Number of Examples** | More examples → more specific instructions | 5–10 diverse examples |
| **Example Diversity** | Diverse examples → more general instructions | Cover edge cases |
| **Example Ordering** | Can influence generated instruction focus | Place typical examples first |
| **Generation Temperature** | Higher → more diverse candidates | T=0.7–1.0 for variety |
| **Model Capability** | Larger models abstract better | GPT-4 class preferred |
Instruction Induction is **the cognitive foundation of automated prompt engineering** — enabling language models to observe, abstract, and articulate task definitions from demonstrations alone, transforming the process of creating effective prompts from a manual authoring challenge into an automated inference problem that scales across unlimited tasks.