winobias

**WinoBias** is the **coreference-resolution bias benchmark that tests whether models rely on gender stereotypes when resolving ambiguous pronouns** - it measures fairness in occupation-gender association reasoning. **What Is WinoBias?** - **Definition**: Dataset of pronoun resolution examples designed to expose gendered occupational bias. - **Task Structure**: Sentences contain occupation terms and pronouns where correct resolution may conflict with stereotype. - **Evaluation Signal**: Performance gap between pro-stereotypical and anti-stereotypical cases. - **Model Scope**: Applicable to language understanding and generation systems with coreference behavior. **Why WinoBias Matters** - **Stereotype Sensitivity**: Detects whether models default to biased gender assumptions. - **Fairness Insight**: Highlights representational harms in linguistic reasoning tasks. - **Mitigation Tracking**: Useful for measuring debiasing effect on pronoun resolution behavior. - **Comparative Value**: Enables cross-model evaluation on a targeted bias mechanism. - **Deployment Relevance**: Coreference bias can propagate into downstream application outputs. **How It Is Used in Practice** - **Gap Measurement**: Compare error rates across stereotype-consistent and stereotype-inconsistent sets. - **Intervention Testing**: Re-evaluate after counterfactual augmentation and debias fine-tuning. - **Holistic Assessment**: Combine with open-ended generation benchmarks for broader fairness coverage. WinoBias is **a focused benchmark for gender stereotype effects in coreference reasoning** - pronoun-resolution disparity analysis provides a clear signal of fairness weaknesses in language models.