Causal Inference and Machine Learning

# Causal Inference and Machine Learning

## Introduction & Motivation

Causal inference distinguishes correlation from causation, critical for decision-making in materials processing, pharmaceutical development, and complex systems. ML combined with causal frameworks enables discovery of true relationships and optimal interventions.

Motivation: Infer causal effects from observational and experimental data.

Applications: Treatment effect estimation, policy evaluation, intervention optimization, mechanism discovery.

---

## Core Concepts & Theory

### Causal DAGs

Directed acyclic graphs.

### Confounding

Spurious associations.

### Identification

Estimability conditions.

### Treatment Effects

Average and heterogeneous effects.

---

## Mathematical Formulation

Potential Outcomes:
$$Y_i(1), Y_i(0) \quad ext{for treatment and control}$$

Average Treatment Effect:
$$ ext{ATE} = \mathbb{E}[Y_i(1) - Y_i(0)]$$

Backdoor Adjustment:
$$P(y|do(x)) = \sum_z P(y|x,z)P(z)$$

---

## Advanced Theory & Extensions

### Double Machine Learning

Debiased estimation.

### Causal Forests

Heterogeneous treatment effects.

### Instrumental Variables

Handling unmeasured confounding.

---

## Computational Considerations

Matching: O(N·D) distance.

Regression: O(N·D²) estimation.

Forest: O(N·log N·T).

---

## Practical Implementation Strategies

### Confounder Control

Regression, matching, weighting.

### Sensitivity Analysis

Robustness to hidden confounding.

### Effect Heterogeneity

Subgroup analysis.

---

## Benchmark Datasets & Evaluation

LaLonde Dataset: Classic treatment study.

IHDP: Causal inference benchmark.

Twins Dataset: Natural experiments.

---

## Key Challenges & Limitations

### Identifiability

Unverifiable assumptions.

### Hidden Confounding

Unmeasured variables.

### Extrapolation

Policy relevance.

---

## Hyperparameter Tuning

Caliper: Matching distance.

Regularization: Effect estimation.

Forest depth: Treatment heterogeneity.

---

## Real-World Applications & Case Studies

Clinical Trials: Treatment efficacy.

Policy Evaluation: Program impact.

Materials: Process optimization.

---

## Integration with Other Methods

Causal inference + ML; + experimental design; + optimization.

---

## Summary & Key Takeaways

Causal inference enables decision-making from data.

Principles:
1. DAGs: Model causality.
2. Confounding: Control spurious.
3. Identification: Assumptions matter.
4. Estimation: Multiple methods.
5. Sensitivity: Test robustness.

---

## Appendix: Practical Labs

### Lab 1: Propensity Score Matching

import numpy as np

def propensity_score_matching(X, T):
 """Estimate propensity score"""
 # Logistic regression (simplified)
 prob_t = 1 / (1 + np.exp(-X @ np.random.randn(X.shape[1])))
 return prob_t

def match_on_propensity(X, T, y, caliper=0.1):
 """Match on propensity score"""
 ps = propensity_score_matching(X, T)
 
 matched_diff = []
 
 for i in np.where(T == 1)[0]:
 control_matches = np.where((T == 0) & (np.abs(ps - ps[i]) < caliper))[0]
 
 if len(control_matches) > 0:
 j = control_matches[0]
 matched_diff.append(y[i] - y[j])
 
 ate = np.mean(matched_diff) if matched_diff else 0
 return ate

X = np.random.randn(100, 5)
T = (X[:, 0] > 0).astype(int)
y = T * 2 + X[:, 0] + np.random.randn(100) * 0.5

ate = match_on_propensity(X, T, y)
print(f"✓ ATE via matching: {ate:.3f}")

### Lab 2: Backdoor Adjustment

import numpy as np

def backdoor_adjustment(X, T, y, Z):
 """Estimate effect controlling for confounders"""
 # Simple regression adjustment
 from sklearn.linear_model import LinearRegression
 
 model = LinearRegression()
 features = np.column_stack([T, Z])
 model.fit(features, y)
 
 # Effect estimate
 effect = model.coef_[0]
 return effect

X = np.random.randn(50, 1)
Z = np.random.randn(50, 2) # Confounders
T = (Z[:, 0] > 0).astype(int)
y = T * 3 + Z[:, 0] + np.random.randn(50) * 0.5

effect = backdoor_adjustment(X, T, y, Z)
print(f"✓ ATE via backdoor: {effect:.3f}")

### Lab 3: Instrumental Variables

import numpy as np

def instrumental_variable_estimation(Z, T, y):
 """IV estimation"""
 # Two-stage least squares
 
 # First stage: predict T from Z
 beta_1 = np.linalg.lstsq(Z.reshape(-1, 1), T, rcond=None)[0]
 T_pred = Z * beta_1
 
 # Second stage: predict y from predicted T
 beta_2 = np.linalg.lstsq(T_pred.reshape(-1, 1), y, rcond=None)[0]
 
 return beta_2[0]

Z = np.random.randn(100) # Instrument
T = 0.5 * Z + np.random.randn(100) * 0.5
y = 2 * T + np.random.randn(100) * 0.5

effect = instrumental_variable_estimation(Z, T, y)
print(f"✓ IV estimate: {effect:.3f}")

### Lab 4: Causal Forest

import numpy as np

class CausalForest:
 def __init__(self, n_trees=100):
 self.n_trees = n_trees
 self.trees = []
 
 def fit(self, X, T, y):
 """Fit causal forest"""
 for _ in range(self.n_trees):
 # Bootstrap sample
 indices = np.random.choice(len(X), len(X), replace=True)
 X_boot = X[indices]
 T_boot = T[indices]
 y_boot = y[indices]
 
 # Fit tree
 tree_effect = np.mean(y_boot[T_boot == 1]) - np.mean(y_boot[T_boot == 0])
 self.trees.append(tree_effect)
 
 def predict(self, X_test):
 """Predict treatment effect"""
 effects = np.array(self.trees)
 return np.mean(effects) * np.ones(len(X_test))

X = np.random.randn(100, 5)
T = np.random.binomial(1, 0.5, 100)
y = T * 2 + X[:, 0] + np.random.randn(100) * 0.5

cf = CausalForest(n_trees=100)
cf.fit(X, T, y)
effects = cf.predict(X[:10])

print(f"✓ Causal forest predictions: {effects[:3]}")

---