Fairness and Bias in Machine Learning
# Fairness and Bias in Machine Learning
## Introduction & Motivation
Fairness: ensuring equitable outcomes across demographics. Bias: systematic errors affecting certain groups. Critical for ethical AI deployment.
Motivation: Build unbiased, equitable machine learning systems.
Applications: Hiring, lending, criminal justice, healthcare.
---
## Core Concepts & Theory
### Demographic Parity
Equal outcome rates across groups.
### Equalized Odds
Equal true positive rates across groups.
### Individual Fairness
Treat similar individuals similarly.
### Causal Fairness
Fairness accounting for causality.
---
## Mathematical Formulation
Demographic Parity:
$$P(\hat{y}=1|A=0) = P(\hat{y}=1|A=1)$$
Equalized Odds:
$$P(\hat{y}=1|Y=1,A=0) = P(\hat{y}=1|Y=1,A=1)$$
Fairness-Accuracy Tradeoff:
$$ ext{Fair Loss} = L_{ ext{accuracy}} + \lambda L_{ ext{fairness}}$$
---
## Advanced Theory & Extensions
### Disparate Impact
Statistical measurement of unfairness.
### Causal Graphs
Fairness through causal models.
### Intersectionality
Multiple group memberships.
---
## Computational Considerations
Bias Detection: O(N·D).
Fairness Enforcement: O(N·D) training overhead.
Certification: O(N²) for worst-case.
---
## Practical Implementation Strategies
### Data Augmentation
Balance group representation.
### Reweighting
Adjust sample weights.
### Threshold Tuning
Group-specific decision thresholds.
---
## Benchmark Datasets & Evaluation
COMPAS: Criminal justice recidivism.
Adult: Income prediction.
German Credit: Loan approval.
---
## Key Challenges & Limitations
### Fairness Definitions
Multiple incompatible notions.
### Accuracy-Fairness Tradeoff
Often in conflict.
### Measurement
Incomplete group information.
---
## Hyperparameter Tuning
Fairness penalty: 0.1-1.0.
Group balance: 0.5 (50-50 split).
Threshold offset: -0.1 to 0.1.
---
## Real-World Applications & Case Studies
Criminal Justice: COMPAS algorithm fairness.
Hiring: Resume screening bias.
Healthcare: Treatment disparity analysis.
---
## Integration with Other Methods
Fairness + interpretability; + robustness; + uncertainty estimation.
---
## Summary & Key Takeaways
Fairness ensures equitable AI systems.
Principles:
1. Equity: Equal outcomes.
2. Parity: Group balance.
3. Odds: Equal true positive rates.
4. Causality: Causal fairness analysis.
5. Tradeoff: Acknowledge accuracy-fairness tension.
---
## Appendix: Practical Labs
### Lab 1: Demographic Parity Check
import numpy as np
def check_demographic_parity(predictions, sensitive_attribute, threshold=0.05):
"""Check if model satisfies demographic parity"""
groups = np.unique(sensitive_attribute)
positive_rates = {}
for group in groups:
mask = sensitive_attribute == group
positive_rate = np.mean(predictions[mask])
positive_rates[group] = positive_rate
# Check parity
max_rate = max(positive_rates.values())
min_rate = min(positive_rates.values())
disparity = max_rate - min_rate
is_fair = disparity <= threshold
return positive_rates, disparity, is_fair
# Simulated predictions and demographics
predictions = np.random.rand(100) > 0.5
demographics = np.random.randint(0, 2, 100) # Binary attribute
rates, disparity, fair = check_demographic_parity(predictions, demographics)
print(f"✓ Demographic parity: disparity={disparity:.3f}, fair={fair}")### Lab 2: Equalized Odds
import numpy as np
def check_equalized_odds(predictions, labels, sensitive_attribute):
"""Check equalized odds across groups"""
groups = np.unique(sensitive_attribute)
tpr_by_group = {}
fpr_by_group = {}
for group in groups:
mask = sensitive_attribute == group
# True positive rate
tp = np.sum((predictions[mask] == 1) & (labels[mask] == 1))
p = np.sum(labels[mask] == 1)
tpr = tp / (p + 1e-8)
# False positive rate
fp = np.sum((predictions[mask] == 1) & (labels[mask] == 0))
n = np.sum(labels[mask] == 0)
fpr = fp / (n + 1e-8)
tpr_by_group[group] = tpr
fpr_by_group[group] = fpr
# Check equalized odds
tpr_disparity = max(tpr_by_group.values()) - min(tpr_by_group.values())
fpr_disparity = max(fpr_by_group.values()) - min(fpr_by_group.values())
return tpr_by_group, fpr_by_group, tpr_disparity, fpr_disparity
predictions = np.random.rand(100) > 0.5
labels = np.random.rand(100) > 0.5
demographics = np.random.randint(0, 2, 100)
tpr, fpr, tpr_d, fpr_d = check_equalized_odds(predictions, labels, demographics)
print(f"✓ Equalized odds: TPR disparity={tpr_d:.3f}, FPR disparity={fpr_d:.3f}")### Lab 3: Bias Mitigation via Reweighting
import numpy as np
def compute_fair_weights(labels, sensitive_attribute):
"""Compute reweighting for demographic parity"""
groups = np.unique(sensitive_attribute)
weights = np.ones(len(labels))
for group in groups:
mask = sensitive_attribute == group
group_size = np.sum(mask)
# Weight to equalize group representation
target_weight = 1.0 / len(groups)
group_weight = target_weight / (group_size / len(labels))
weights[mask] = group_weight
# Normalize
weights = weights / np.sum(weights) * len(weights)
return weights
labels = np.random.rand(100) > 0.5
demographics = np.concatenate([np.zeros(60), np.ones(40)])
weights = compute_fair_weights(labels, demographics)
print(f"✓ Fair weights: sum={np.sum(weights):.1f}, mean={np.mean(weights):.3f}")### Lab 4: Fair Model Training
import numpy as np
class FairModelTrainer:
def __init__(self, model_dim=10, fairness_weight=0.5):
self.model = np.random.randn(model_dim, 1) * 0.01
self.fairness_weight = fairness_weight
def compute_accuracy_loss(self, predictions, labels):
"""Standard classification loss"""
accuracy_loss = np.mean((predictions - labels) ** 2)
return accuracy_loss
def compute_fairness_loss(self, predictions, sensitive_attr):
"""Fairness loss: minimize group disparities"""
groups = np.unique(sensitive_attr)
group_losses = []
for group in groups:
mask = sensitive_attr == group
group_pred = predictions[mask]
group_mean = np.mean(group_pred)
group_losses.append(group_mean)
# Variance across groups
fairness_loss = np.var(group_losses)
return fairness_loss
def train_fair_model(self, features, labels, sensitive_attr, epochs=10, lr=0.01):
"""Train model with fairness constraint"""
for epoch in range(epochs):
# Predictions
predictions = features @ self.model
# Compute losses
acc_loss = self.compute_accuracy_loss(predictions, labels)
fair_loss = self.compute_fairness_loss(predictions, sensitive_attr)
# Combined loss
total_loss = acc_loss + self.fairness_weight * fair_loss
# Update (simplified)
self.model += lr * np.random.randn(*self.model.shape) * 0.001
if epoch % 3 == 0:
print(f"Epoch {epoch}: accuracy_loss={acc_loss:.3f}, fairness_loss={fair_loss:.3f}")
trainer = FairModelTrainer(fairness_weight=0.5)
features = np.random.randn(100, 10)
labels = np.random.rand(100) > 0.5
demographics = np.random.randint(0, 2, 100)
trainer.train_fair_model(features, labels, demographics)
print(f"✓ Fair model trained")---