answer relevance

**Answer relevance** is the **evaluation of how directly and completely a model response addresses the user intent and requested scope** - it captures usefulness from the end-user perspective. **What Is Answer relevance?** - **Definition**: Fit between produced answer content and the explicit or implicit user question. - **Evaluation Dimension**: Considers topical alignment, scope match, and response completeness. - **Common Failure Modes**: Off-topic details, partial answers, and overlong digressions. - **Relation to Grounding**: An answer can be faithful to context yet still not answer the user well. **Why Answer relevance Matters** - **User Satisfaction**: Relevance is a direct driver of perceived assistant quality. - **Task Completion**: High relevance reduces follow-up turns and clarification overhead. - **Operational Value**: Business workflows need actionable answers aligned to intent. - **Evaluation Balance**: Complements factuality metrics for a complete quality picture. - **Product Iteration**: Relevance errors reveal prompt design and routing weaknesses. **How It Is Used in Practice** - **Intent-Aware Rubrics**: Score whether answers cover required constraints and requested detail level. - **Human Plus Model Judges**: Combine evaluator models with sampled human review for calibration. - **Prompt Refinement**: Tune instruction templates to prioritize concise intent fulfillment. Answer relevance is **a core outcome metric for real-world assistant utility** - strong answer relevance ensures grounded responses are not only correct but useful.