Fairness Bias in Machine Learning
# Fairness & Bias in Machine Learning
## Introduction & Motivation
Fairness: ensure ML systems don't discriminate. Bias detection and mitigation. Applications: hiring, lending, criminal justice.
Motivation: Build equitable, non-discriminatory systems.
Applications: Hiring systems, loan decisions, predictive policing.
---
## Core Concepts & Theory
### Demographic Parity
Equal outcome rates across groups.
### Equalized Odds
Equal true positive rates across groups.
### Disparate Impact
Statistical evidence of discrimination.
### Fairness Constraints
Incorporate fairness into optimization.
---
## Mathematical Formulation
Demographic Parity:
$$P(\hat{Y}=1 | A=0) = P(\hat{Y}=1 | A=1)$$
Equalized Odds:
$$P(\hat{Y}=1 | A=0, Y=1) = P(\hat{Y}=1 | A=1, Y=1)$$
Disparate Impact Ratio:
$$\frac{ ext{selection rate}_{ ext{protected}}}{ ext{selection rate}_{ ext{unprotected}}} \geq 0.8$$
---
## Advanced Theory & Extensions
### Counterfactual Fairness
Fairness through causal reasoning.
### Fairness-Accuracy Trade-offs
Pareto frontier exploration.
### Individual Fairness
Similar individuals treated similarly.
---
## Computational Considerations
Bias measurement: O(N).
Constraint enforcement: O(N·iterations).
Trade-off computation: O(models).
---
## Practical Implementation Strategies
### Preprocessing
Remove or adjust biased features.
### Inprocessing
Fairness constraints during training.
### Postprocessing
Adjust predictions for fairness.
---
## Benchmark Datasets & Evaluation
Adult Dataset: Income prediction benchmark.
COMPAS: Criminal justice recidivism.
German Credit: Credit approval fairness.
---
## Key Challenges & Limitations
### Fairness Definitions
Multiple notions of fairness.
### Causality Identification
Determine discrimination sources.
### Trade-offs
Fairness vs. accuracy vs. other metrics.
---
## Hyperparameter Tuning
Protected attribute weight: 0.1-1.0.
Fairness constraint strength: 0.01-1.0.
Threshold (disparate impact): 0.6-0.9.
---
## Real-World Applications & Case Studies
Hiring: Remove gender bias in screening.
Lending: Ensure fair credit decisions.
Criminal Justice: Reduce recidivism bias.
---
## Integration with Other Methods
Fairness + interpretability for bias diagnosis; + causal inference for mechanism understanding.
---
## Summary & Key Takeaways
Fairness in ML via demographic parity and constrained optimization enables non-discriminatory systems.
Principles:
1. Demographic parity: Equal outcomes.
2. Equalized odds: Equal true positives.
3. Bias measurement: Quantify discrimination.
4. Constraint enforcement: Incorporate fairness.
5. Trade-off exploration: Accuracy-fairness balance.
---
---
## Appendix: Practical Labs
### Lab 1: Demographic Parity
import numpy as np
def compute_demographic_parity(predictions, protected_attr):
"""Measure demographic parity violation"""
# Prediction rates per group
group_0 = predictions[protected_attr == 0]
group_1 = predictions[protected_attr == 1]
rate_0 = np.mean(group_0)
rate_1 = np.mean(group_1)
# Parity violation
violation = np.abs(rate_0 - rate_1)
return violation, rate_0, rate_1
# Test
np.random.seed(42)
preds = np.random.randint(0, 2, 100)
attr = np.random.randint(0, 2, 100)
violation, r0, r1 = compute_demographic_parity(preds, attr)
assert 0 <= violation <= 1, "Violation in range"
print("✓ Demographic parity working")
if __name__ == "__main__":
print("Lab 1: DemographicParity - PASSED")### Lab 2: Equalized Odds
import numpy as np
def compute_equalized_odds(predictions, true_labels, protected_attr):
"""Measure equalized odds violation"""
# True positive rates per group (for positive class)
tpr_0 = np.mean(predictions[(true_labels == 1) & (protected_attr == 0)])
tpr_1 = np.mean(predictions[(true_labels == 1) & (protected_attr == 1)])
# False positive rates per group (for negative class)
fpr_0 = np.mean(predictions[(true_labels == 0) & (protected_attr == 0)])
fpr_1 = np.mean(predictions[(true_labels == 0) & (protected_attr == 1)])
# Violation
tpr_violation = np.abs(tpr_0 - tpr_1)
fpr_violation = np.abs(fpr_0 - fpr_1)
return tpr_violation, fpr_violation
# Test
np.random.seed(42)
preds = np.random.randint(0, 2, 100)
labels = np.random.randint(0, 2, 100)
attr = np.random.randint(0, 2, 100)
tpr_viol, fpr_viol = compute_equalized_odds(preds, labels, attr)
assert 0 <= tpr_viol <= 1, "TPR violation in range"
assert 0 <= fpr_viol <= 1, "FPR violation in range"
print("✓ Equalized odds working")
if __name__ == "__main__":
print("Lab 2: EqualizedOdds - PASSED")### Lab 3: Disparate Impact
import numpy as np
def compute_disparate_impact_ratio(predictions, protected_attr):
"""Compute disparate impact ratio"""
# Selection rate per group
selected_0 = np.mean(predictions[protected_attr == 0])
selected_1 = np.mean(predictions[protected_attr == 1])
# Disparate impact ratio
ratio = selected_0 / (selected_1 + 1e-8)
return ratio, selected_0, selected_1
# Test
np.random.seed(42)
preds = np.random.randint(0, 2, 100)
attr = np.random.randint(0, 2, 100)
ratio, sel0, sel1 = compute_disparate_impact_ratio(preds, attr)
assert ratio >= 0, "Ratio non-negative"
print("✓ Disparate impact working")
if __name__ == "__main__":
print("Lab 3: DisparateImpact - PASSED")### Lab 4: Fairness Constraint
import numpy as np
def apply_fairness_constraint(predictions, targets, protected_attr, fairness_weight=0.5):
"""Apply fairness-accuracy trade-off"""
# Accuracy loss
accuracy_loss = np.mean(predictions != targets)
# Fairness loss (demographic parity)
group_0 = predictions[protected_attr == 0]
group_1 = predictions[protected_attr == 1]
rate_0 = np.mean(group_0)
rate_1 = np.mean(group_1)
fairness_loss = np.abs(rate_0 - rate_1)
# Combined loss
total_loss = (1 - fairness_weight) * accuracy_loss + fairness_weight * fairness_loss
return total_loss
# Test
np.random.seed(42)
preds = np.random.randint(0, 2, 100)
targets = np.random.randint(0, 2, 100)
attr = np.random.randint(0, 2, 100)
loss = apply_fairness_constraint(preds, targets, attr, fairness_weight=0.5)
assert np.isfinite(loss), "Loss finite"
print("✓ Fairness constraint working")
if __name__ == "__main__":
print("Lab 4: FairnessConstraint - PASSED")