fairness metrics

Fairness metrics quantify and measure bias across demographic groups to enable evaluation and improvement. **Key metrics**: **Demographic parity**: Equal positive prediction rates across groups. **Equalized odds**: Equal true positive and false positive rates. **Equal opportunity**: Equal true positive rates only. **Predictive parity**: Equal precision across groups. **Individual fairness**: Similar individuals get similar predictions. **Group-level analysis**: Slice performance metrics by demographic attributes - accuracy, precision, recall per group. **Impossibility results**: Some fairness metrics are mathematically incompatible - can't satisfy all simultaneously. **Selection criteria**: Choose metrics based on context, harm model, stakeholder input. **NLP-specific**: Representation analysis in embeddings, stereotype association tests (WEAT, SEAT), task performance across dialects/demographics. **Benchmarks**: BBQ, StereoSet, WinoBias, CrowS-Pairs. **Reporting**: Model cards should include fairness evaluation, disaggregated metrics. **Challenges**: Demographic data often unavailable, intersectionality, proxy measures. **Best practices**: Multiple metrics, qualitative + quantitative evaluation, ongoing monitoring. Foundation for bias auditing and mitigation.