rouge score

ROUGE (Recall-Oriented Understudy for Gisting Evaluation) is a family of recall-based automatic evaluation metrics primarily designed for summarization quality assessment, measuring the overlap between a generated summary and reference summaries with an emphasis on how much of the reference content is captured by the candidate. Introduced by Lin in 2004, ROUGE complements BLEU's precision focus by measuring recall — while BLEU asks "what fraction of the candidate was correct?" ROUGE asks "what fraction of the reference was captured?" ROUGE includes several variants: ROUGE-N measures n-gram recall (ROUGE-1 for unigram overlap, ROUGE-2 for bigram overlap — ROUGE-2 is particularly popular as it captures some word ordering), ROUGE-L uses the Longest Common Subsequence (LCS) between candidate and reference (capturing sentence-level structure without requiring consecutive matches — subsequences allow gaps), ROUGE-W is a weighted version of ROUGE-L that favors consecutive matches over fragmented ones, ROUGE-S measures skip-bigram co-occurrence (pairs of words in their sentence order with arbitrary gaps between them — capturing long-range content overlap), and ROUGE-SU adds unigram counting to skip-bigrams. For each variant, ROUGE computes recall (R), precision (P), and F-measure (F1 = 2PR/(P+R)), though recall was originally emphasized for summarization (ensuring summaries cover important content). ROUGE scores typically range from 0 to 1, with ROUGE-1 F1 scores for modern summarization systems ranging from 0.40-0.50 on CNN/DailyMail. Strengths include: intuitive interpretation (higher recall means more reference content captured), fast computation enabling large-scale evaluation, multiple variants capturing different overlap aspects, and strong corpus-level correlation with human judgments for extractive summarization. Limitations include: insensitivity to factual correctness (generated text with wrong facts can score highly if it shares many n-grams with references), poor evaluation of abstractive summaries (novel phrasing penalized), and dependence on reference quality and quantity.