stereoset

**StereoSet** is the **bias benchmark that evaluates whether language models prefer stereotypical completions over anti-stereotypical or unrelated alternatives** - it measures stereotype tendency while accounting for language-modeling quality. **What Is StereoSet?** - **Definition**: Evaluation dataset with contexts paired to stereotype, anti-stereotype, and unrelated continuation options. - **Target Dimensions**: Includes social categories such as gender, race, religion, and profession. - **Scoring Concept**: Separates stereotype preference from general language fluency performance. - **Evaluation Use**: Quantifies tendency to choose or assign higher likelihood to stereotyped content. **Why StereoSet Matters** - **Bias Visibility**: Provides direct signal of stereotype preference behavior in language models. - **Balanced Assessment**: Avoids conflating fairness with raw language-model quality alone. - **Benchmark Utility**: Widely used in fairness studies and mitigation comparisons. - **Intervention Feedback**: Helps assess whether debiasing changes stereotype tendency. - **Release Governance**: Useful as one component in fairness evaluation suites. **How It Is Used in Practice** - **Model Scoring**: Compute benchmark outputs on held-out model versions. - **Trend Analysis**: Compare stereotype-related metrics before and after mitigation updates. - **Portfolio Evaluation**: Combine with other fairness benchmarks for broader risk coverage. StereoSet is **an important benchmark for stereotype bias measurement in LLMs** - it offers structured evidence on how strongly models favor stereotyped continuations under controlled prompts.