disparate impact

**Disparate impact** is a legal and fairness concept describing a situation where a model, algorithm, or policy **disproportionately affects** one demographic group compared to another, even if the system appears **facially neutral** — meaning it doesn't explicitly use protected attributes like race or gender. **Legal Origin** - Rooted in **US employment discrimination law** (Civil Rights Act, Griggs v. Duke Power, 1971). - The **four-fifths (80%) rule**: If the selection rate for a protected group is less than **80%** of the rate for the most-selected group, there is evidence of disparate impact. - Example: If 60% of male applicants are hired but only 40% of female applicants, the ratio is 40/60 = 67% < 80%, indicating potential disparate impact. **Disparate Impact in AI/ML** - **Proxy Variables**: Even without explicit use of race or gender, models can learn to use **correlated features** (zip code, name, browsing history) as proxies that produce discriminatory outcomes. - **Training Data Bias**: Models trained on historically biased data will learn and reproduce those biases. - **Feature Engineering**: Seemingly neutral features can encode social inequalities. **Examples in AI** - **Credit Scoring**: A model that denies loans more often to people from certain zip codes may disproportionately affect racial minorities due to historical residential segregation. - **Hiring Algorithms**: Resume screening tools trained on historical hiring data may penalize female applicants in male-dominated industries. - **Facial Recognition**: Higher error rates for darker-skinned individuals compared to lighter-skinned individuals. - **Healthcare**: Clinical algorithms that use cost as a proxy for need can disadvantage groups with less access to healthcare. **Measuring Disparate Impact** - **Adverse Impact Ratio**: Selection rate of disadvantaged group / selection rate of advantaged group. - **Statistical Parity Difference**: Difference in positive outcome rates between groups. - **Intersectional Analysis**: Check for disparate impact across **combinations** of protected attributes. **Regulatory Landscape** Disparate impact analysis is increasingly required by AI regulations, including the **EU AI Act**, **NYC Local Law 144** (automated employment decision tools), and **EEOC guidelines**.