self-evaluation
**Self-evaluation** in AI refers to a model's ability to **assess, critique, and score its own outputs**. This metacognitive capability enables language models to identify errors, rate confidence, and improve responses through self-reflection, without requiring external feedback.
**Common Self-Evaluation Approaches**
- **Self-Scoring**: Ask the model to rate its own response on a scale (e.g., "Rate the accuracy of your answer from 1-10 and explain why").
- **Self-Verification**: Generate a response, then prompt the model to check it for factual errors, logical inconsistencies, or missing information.
- **Self-Critique**: Ask the model to identify weaknesses in its own output and suggest improvements.
- **Consistency Checking**: Generate multiple responses and check whether they agree — inconsistency signals potential errors.
**Applications**
- **Constitutional AI**: Anthropic's approach uses self-critique against a set of principles to improve safety without human labels.
- **Self-Refine**: Generate → critique → revise loop that iteratively improves response quality.
- **Confidence Estimation**: The model's self-assessed confidence can flag responses that need human review.
- **Best-of-N Selection**: Generate N responses, have the model score each, and return the highest-rated one.
**Limitations**
- **Overconfidence**: Models are often poorly calibrated — they may rate incorrect answers highly because they "sound right."
- **Blind Spots**: A model that makes an error due to a knowledge gap will also fail to detect that error in self-evaluation.
- **Sycophantic Self-Assessment**: Models tend to rate their own outputs favorably, especially when the evaluation prompt doesn't explicitly encourage criticism.
- **Inconsistency**: Self-evaluation scores can vary significantly across runs or phrasings.
**When It Works Well**
Self-evaluation is most reliable for detecting **format errors**, **logical contradictions**, and **internally inconsistent claims**. It is least reliable for **factual accuracy** verification, where the model may confidently confirm its own hallucinations.