mean average precision (map)

**Mean Average Precision (MAP)** is the **average of Average Precision across multiple queries** — the standard metric for evaluating search and retrieval systems across entire query sets, providing a single score for overall system performance. **What Is MAP?** - **Definition**: Mean of Average Precision scores across all queries. - **Formula**: MAP = (Σ AP(q)) / |Q| where Q is set of queries. - **Range**: 0 (worst) to 1 (perfect). **How MAP Works** **1. For each query, compute Average Precision (AP)**. **2. Average AP scores across all queries**. **Example** Query 1: AP = 0.8. Query 2: AP = 0.6. Query 3: AP = 0.9. - MAP = (0.8 + 0.6 + 0.9) / 3 = 0.77. **Why MAP?** - **Standard Metric**: Most widely used for IR evaluation. - **Comprehensive**: Evaluates entire system across all queries. - **Position-Aware**: Rewards relevant results at top. - **Recall-Aware**: Considers all relevant items. - **Single Score**: Easy to compare systems. **MAP@K**: Compute MAP considering only top-K results per query. **Advantages** - **Industry Standard**: Used in TREC, academic IR research. - **Comprehensive**: Captures precision, recall, and position. - **Comparable**: Single score for system comparison. **Limitations** - **Binary Relevance**: Doesn't handle graded relevance (use NDCG). - **Query Averaging**: Treats all queries equally (may want weighted). - **Requires Relevance Judgments**: Need labeled data for all queries. **MAP vs. Other Metrics** **vs. NDCG**: MAP binary relevance, NDCG graded relevance. **vs. MRR**: MAP considers all relevant, MRR only first. **vs. Precision@K**: MAP comprehensive, P@K single cutoff. **Applications**: Search engine evaluation, information retrieval research, recommendation system evaluation, document retrieval. **Tools**: trec_eval (standard IR evaluation tool), scikit-learn, IR libraries. MAP is **the gold standard for IR evaluation** — by averaging precision across all relevant positions and all queries, MAP provides the most comprehensive single-number assessment of search and retrieval system quality.