fairness

**AI Fairness** is the **interdisciplinary field that develops metrics, methods, and interventions to ensure AI systems do not produce discriminatory outcomes for protected groups — based on race, gender, age, disability, religion, or other characteristics** — addressing both the technical challenge of measuring bias and the sociotechnical challenge of defining what "fair" means across competing stakeholder interests. **What Is AI Fairness?** - **Definition**: The set of principles, metrics, and mitigation techniques ensuring that AI systems' predictions, decisions, and outcomes do not unfairly disadvantage individuals based on protected characteristics — and that the benefits and harms of AI are equitably distributed across demographic groups. - **Regulated Domains**: Credit (Equal Credit Opportunity Act), hiring (Equal Employment Opportunity), housing (Fair Housing Act), healthcare, criminal justice (risk assessment), and any automated decision affecting individuals. - **Challenge**: Fairness is not a single mathematical property — there are dozens of competing formal definitions, and satisfying multiple definitions simultaneously is often mathematically impossible. - **Sociotechnical Nature**: Technical fairness metrics are necessary but insufficient — defining "fair" requires normative judgments about values, history, and social goals that extend beyond machine learning. **Why AI Fairness Matters** - **Documented Harms**: COMPAS recidivism algorithm: false positive rate 2x higher for Black defendants than white. Amazon recruiting tool: systematically downrated women's resumes. Healthcare algorithm: Black patients received worse care recommendations due to cost proxy for need. - **Regulatory Compliance**: EU AI Act classifies high-risk AI (credit, employment, justice) with mandatory fairness documentation requirements. US agencies issue guidance on AI fairness for regulated industries. - **Societal Trust**: AI systems that systematically disadvantage protected groups erode public trust in both AI and the institutions deploying it. - **Business Risk**: Discriminatory AI creates legal liability, reputational damage, and regulatory penalties — fairness is a business imperative, not only an ethical one. - **Feedback Loops**: Biased AI predictions shape future data — if a model under-approves loans in a neighborhood, the neighborhood receives less investment, confirming the model's discriminatory prediction. **Sources of Bias** **Historical Bias**: - The world reflects historical discrimination — training data encodes past prejudice. - Example: CEOs in historical data are predominantly male → AI associates "CEO" with male features. - Mitigations: Re-weighting, counterfactual data augmentation, targeted data collection. **Representation Bias**: - Training data under-represents certain populations — model performs worse on underrepresented groups. - Example: Facial recognition trained mostly on light-skinned faces → 34% error rate for dark-skinned women vs. 0.8% for light-skinned men (Buolamwini & Gebru, 2018). - Mitigations: Stratified sampling, targeted data collection, evaluation by subgroup. **Measurement Bias**: - Proxy variables encode protected attributes — even without using race directly, using zip code or name introduces racial information. - Example: Using zip code as a feature encodes racial segregation patterns. - Mitigations: Fairness-aware feature selection, adversarial debiasing. **Label Bias**: - Human-generated labels encode annotator biases. - Example: Annotators systematically rate identical resumes lower when names appear female. - Mitigations: Inter-annotator agreement audits, diverse annotator pools, blind annotation. **Aggregation Bias**: - A model trained on aggregated data may not perform well for any subgroup. - Example: A diabetes risk model trained on combined demographics may underperform for Hispanic women if their risk factors differ systematically. **Fairness Metrics** **Group Fairness Metrics**: - **Demographic Parity**: P(Ŷ=1 | A=0) = P(Ŷ=1 | A=1). Positive prediction rate must be equal across groups. Does not account for genuine differences in base rates. - **Equalized Odds**: P(Ŷ=1 | Y=1, A=0) = P(Ŷ=1 | Y=1, A=1) AND P(Ŷ=1 | Y=0, A=0) = P(Ŷ=1 | Y=0, A=1). True positive rates AND false positive rates must be equal across groups. Most commonly required in high-stakes settings. - **Equal Opportunity**: P(Ŷ=1 | Y=1, A=0) = P(Ŷ=1 | Y=1, A=1). True positive rates equal — minimize false negatives equally across groups. Appropriate when false negatives are the primary harm (missing qualified candidates). - **Calibration**: P(Y=1 | Ŷ=p, A=0) = P(Y=1 | Ŷ=p, A=1) = p. Predicted probabilities reflect true frequencies equally across groups. **The Impossibility Theorem**: Chouldechova (2017) and Kleinberg et al. (2017) proved that demographic parity, equalized odds, and calibration cannot all be simultaneously satisfied when base rates differ across groups — fairness metric choice is a values decision. **Bias Mitigation Approaches** | Phase | Approach | Method | |-------|----------|--------| | Pre-processing | Modify training data | Reweighting, resampling, counterfactual augmentation | | In-processing | Constrain model training | Adversarial debiasing, fairness constraints in loss | | Post-processing | Adjust model outputs | Threshold calibration per group, reject option | AI fairness is **the social contract between AI systems and the communities they affect** — by developing rigorous tools for measuring and mitigating discriminatory outcomes, fairness research ensures that AI's benefits are distributed equitably rather than amplifying historical inequities, making the difference between AI as an engine of opportunity and AI as a force for entrenching systemic discrimination.