mean reciprocal rank

**Mean reciprocal rank** is the **retrieval metric that averages the reciprocal position of the first relevant result across queries** - MRR emphasizes how quickly users encounter a correct hit. **What Is Mean reciprocal rank?** - **Definition**: Average of 1 divided by rank of first relevant item for each query. - **Priority Behavior**: Strongly rewards placing at least one correct result at top positions. - **Task Fit**: Useful for single-answer or first-hit-dominant retrieval scenarios. - **Limitation**: Ignores relevance quality beyond the first relevant result. **Why Mean reciprocal rank Matters** - **Early Success Signal**: Captures user-facing utility when first correct hit is critical. - **Ranking Sharpness**: Penalizes systems that place relevant evidence deep in list. - **Operational Simplicity**: Easy to interpret and compare across retriever variants. - **RAG Alignment**: Strong first-hit ranking improves top context quality for generation. - **Optimization Focus**: Useful objective for first-stage retrieval and rerank tuning. **How It Is Used in Practice** - **Per-Query Diagnostics**: Inspect low reciprocal-rank queries for retrieval failure patterns. - **Metric Portfolio**: Combine MRR with recall and MAP for broader evaluation coverage. - **Release Tracking**: Monitor MRR regressions after index and model updates. Mean reciprocal rank is **a practical first-hit quality metric in retrieval systems** - improving MRR often yields immediate gains in user-perceived relevance and grounded-answer reliability.