iterated amplification

**Iterated Amplification** is an **AI alignment technique that bootstraps human oversight by iteratively using AI assistance to solve increasingly complex evaluation tasks** — starting with problems humans can evaluate directly, then using AI-assisted humans to evaluate slightly harder problems, and continuing to expand the frontier of evaluable tasks. **Amplification Process** - **Base Case**: Human evaluates simple AI outputs directly — standard RLHF. - **Amplification Step**: For harder tasks, decompose into sub-problems that a human-with-AI-assistant can evaluate. - **Iteration**: The AI assistant itself was trained using the previous round's amplified evaluator. - **Distillation**: Train a new model to mimic the amplified evaluator — producing a standalone, efficient model. **Why It Matters** - **Scalable Oversight**: Enables evaluation of AI outputs that are too complex for unaided human judgment. - **Alignment Path**: Provides a concrete path to aligning superhuman AI — evaluation capability grows with AI capability. - **Decomposition**: Complex tasks are decomposed into human-manageable sub-problems — divide and conquer for alignment. **Iterated Amplification** is **growing the evaluator alongside the AI** — bootstrapping human oversight to keep pace with increasingly capable AI systems.