Fairness metrics quantify and measure bias across demographic groups to enable evaluation and improvement. Key metrics: Demographic parity: Equal positive prediction rates across groups. Equalized odds: Equal true positive and false positive rates. Equal opportunity: Equal true positive rates only. Predictive parity: Equal precision across groups. Individual fairness: Similar individuals get similar predictions. Group-level analysis: Slice performance metrics by demographic attributes - accuracy, precision, recall per group. Impossibility results: Some fairness metrics are mathematically incompatible - can't satisfy all simultaneously. Selection criteria: Choose metrics based on context, harm model, stakeholder input. NLP-specific: Representation analysis in embeddings, stereotype association tests (WEAT, SEAT), task performance across dialects/demographics. Benchmarks: BBQ, StereoSet, WinoBias, CrowS-Pairs. Reporting: Model cards should include fairness evaluation, disaggregated metrics. Challenges: Demographic data often unavailable, intersectionality, proxy measures. Best practices: Multiple metrics, qualitative + quantitative evaluation, ongoing monitoring. Foundation for bias auditing and mitigation.
Explore 500+ Semiconductor & AI Topics
From EUV lithography to CUDA optimization — search the full knowledge base or chat with our AI assistant.