Explainability Interpretability Shap Lime Model Explanations
# Explainability & Interpretability: SHAP & LIME Model Explanations
## Introduction & Motivation
Model-agnostic interpretability explains predictions without model retraining. LIME: local linear approximation; perturb input, fit simple model. SHAP: Shapley values; game-theoretic attribution; each feature's contribution. Critical for high-stakes (medical, lending), regulatory compliance, debugging.
Motivation: Black-box models untrustworthy; need explanations. Post-hoc methods don't require model modification.
Applications: Medical diagnosis, loan approval, content moderation, anomaly detection.
---
## Core Concepts & Theory
### LIME
Approximate complex model locally via simple model (linear). Perturb input, get predictions, fit weighted regression.
### SHAP
Shapley values: fair feature attribution. Average marginal contribution across all permutations.
### Feature Importance
Global: average |attribution| over samples. Local: attribution per prediction.
---
## Mathematical Formulation
LIME local approximation:
$$\arg\min_g \sum_i L(f(x), g(x'_i)) w_i(x) + \Omega(g)$$
where g = simple model, w_i = kernel weighting locality.
Shapley value:
$$\phi_i = \frac{1}{|S|!} \sum_{S \subseteq F \setminus \{i\}} [v(S \cup \{i\}) - v(S)]$$
where v(S) = model value with feature set S.
---
## Advanced Theory & Extensions
### TreeSHAP
Efficient SHAP for tree models; O(T L) vs. O(2^d).
### Accumulated Local Effects (ALE)
Avoid collinearity issues in partial dependence; more robust.
### Counterfactual Explanations
Find minimal input changes to change prediction.
---
## Computational Considerations
LIME: O(K × forward passes) for K perturbations.
SHAP: O(2^d) permutations exact; exponential. Use approximations (sampling, TreeSHAP).
Acceleration: Parallel evaluation, caching, GPU.
---
## Practical Implementation Strategies
### Perturbation Strategy
Gaussian noise, feature removal, realistic distributions.
### Number of Samples
Balance fidelity-speed; typically 1000-10000.
### Visualization
Force plots, dependence plots, summary plots.
---
## Benchmark Datasets & Evaluation
Synthetic: Known true importances; measure correlation.
ImageNet: Visual saliency maps; human evaluation.
Tabular: UCI datasets; domain expert validation.
Metrics: Fidelity (match model), sparsity, stability.
---
## Key Challenges & Limitations
### High-Dimensional Data
Curse of dimensionality; many features hard to perturb realistically.
### Computational Cost
SHAP expensive for large datasets/complex models.
### Interpretation Challenges
Feature importance can be misleading; correlation vs. causation.
---
## Hyperparameter Tuning
LIME kernel width: 0.5-1.0 (Gaussian kernel σ).
SHAP samples: 100-10000 for approximation.
Feature interactions: Consider pairwise, higher-order effects.
---
## Real-World Applications & Case Studies
Healthcare: SHAP for risk score interpretability; regulatory requirement.
Finance: LIME for credit decision explanations; compliance.
Fraud Detection: Local importance per transaction; real-time.
---
## Integration with Other Methods
Explainability + Uncertainty → confidence + explanation.
Explainability + Active Learning → explain uncertain predictions.
---
## Summary & Key Takeaways
Explainability via LIME (local linear) and SHAP (Shapley attribution) provides model-agnostic feature importance for predictions.
Principles:
1. LIME: local linear approximation via perturbation + weighting.
2. SHAP: fair attribution via Shapley values.
3. TreeSHAP: efficient for tree models.
4. Global vs. local importance; aggregation strategies.
5. Perturbation strategy, sample size affect fidelity.
---
---
## Appendix: Practical Labs
### Lab 1: LIME Explanation
import numpy as np
from sklearn.linear_model import LinearRegression
from sklearn.datasets import make_classification
def lime_explain(model, x_sample, perturb_fn, n_samples=1000, kernel_width=1.0):
"""LIME: explain prediction via local linear model"""
# Generate perturbations
X_perturb = np.array([perturb_fn(x_sample) for _ in range(n_samples)])
# Get predictions
y_perturb = model.predict(X_perturb)
# Kernel weighting (exponential kernel)
distances = np.linalg.norm(X_perturb - x_sample, axis=1)
weights = np.exp(-(distances**2) / (2 * kernel_width**2))
# Fit local linear model
lime_model = LinearRegression()
lime_model.fit(X_perturb, y_perturb, sample_weight=weights)
# Feature importances: coefficients
importances = np.abs(lime_model.coef_)
return importances, lime_model
# Data
X, y = make_classification(n_samples=100, n_features=10, n_informative=5, random_state=42)
from sklearn.ensemble import RandomForestClassifier
model = RandomForestClassifier(random_state=42)
model.fit(X, y)
x_sample = X[0]
perturb_fn = lambda x: x + np.random.randn(len(x)) * 0.5
importances, lime_model = lime_explain(model, x_sample, perturb_fn)
print(f"Feature importances: {importances[:5]}")
assert len(importances) == 10, "Should have 10 features"
assert (importances >= 0).all(), "Importances should be non-negative"
print("✓ LIME working")
if __name__ == "__main__":
print("Lab 1: LIME - PASSED")### Lab 2: Feature Importance Aggregation
import numpy as np
def aggregate_feature_importance(importances_per_sample, aggregation='mean'):
"""Aggregate local importances to global"""
if aggregation == 'mean':
global_imp = importances_per_sample.mean(axis=0)
elif aggregation == 'median':
global_imp = np.median(importances_per_sample, axis=0)
elif aggregation == 'max':
global_imp = importances_per_sample.max(axis=0)
else:
raise ValueError(f"Unknown aggregation: {aggregation}")
# Normalize
global_imp = global_imp / global_imp.sum()
return global_imp
# Sample per-instance importances
importances_local = np.random.exponential(scale=0.1, size=(50, 10))
global_mean = aggregate_feature_importance(importances_local, 'mean')
global_median = aggregate_feature_importance(importances_local, 'median')
print(f"Global importance (mean): {global_mean[:5]}")
print(f"Global importance (median): {global_median[:5]}")
assert len(global_mean) == 10, "Should have 10 features"
assert np.isclose(global_mean.sum(), 1.0), "Should sum to 1"
print("✓ Feature importance aggregation working")
if __name__ == "__main__":
print("Lab 2: Aggregation - PASSED")### Lab 3: Partial Dependence Plots
import numpy as np
def partial_dependence(model, X, feature_idx, n_grid=20):
"""Compute partial dependence for one feature"""
X_partial = np.tile(X.mean(axis=0), (n_grid * len(X), 1))
# Vary feature_idx
feature_range = np.linspace(X[:, feature_idx].min(), X[:, feature_idx].max(), n_grid)
predictions = []
for val in feature_range:
X_partial_copy = X_partial.copy()
X_partial_copy[:, feature_idx] = val
pred = model.predict(X_partial_copy).mean()
predictions.append(pred)
return feature_range, np.array(predictions)
from sklearn.ensemble import RandomForestClassifier
from sklearn.datasets import make_classification
X, y = make_classification(n_samples=100, n_features=5, random_state=42)
model = RandomForestClassifier(random_state=42)
model.fit(X, y)
feature_range, pd_curve = partial_dependence(model, X, feature_idx=0, n_grid=10)
print(f"Feature range: [{feature_range.min():.2f}, {feature_range.max():.2f}]")
print(f"PD curve length: {len(pd_curve)}")
assert len(pd_curve) == 10, "Should have 10 grid points"
assert all(np.isfinite(p) for p in pd_curve), "All predictions should be finite"
print("✓ Partial dependence working")
if __name__ == "__main__":
print("Lab 3: PDP - PASSED")### Lab 4: Explanation Stability
import numpy as np
from sklearn.ensemble import RandomForestClassifier
from sklearn.datasets import make_classification
def compute_explanation_stability(model, X_sample, perturb_fn, n_runs=10, n_samples_per_run=500):
"""Measure consistency of explanations across runs"""
explanations = []
for _ in range(n_runs):
# Simulate LIME-like explanation
X_perturb = np.array([perturb_fn(X_sample) for _ in range(n_samples_per_run)])
y_perturb = model.predict(X_perturb)
# Feature correlation with outcome
importances = []
for j in range(X_perturb.shape[1]):
corr = np.corrcoef(X_perturb[:, j], y_perturb.astype(float))[0, 1]
importances.append(np.abs(corr) if np.isfinite(corr) else 0)
explanations.append(importances)
explanations = np.array(explanations)
stability = 1 - np.std(explanations, axis=0).mean() # Lower std = higher stability
return stability, explanations
X, y = make_classification(n_samples=100, n_features=5, random_state=42)
model = RandomForestClassifier(random_state=42)
model.fit(X, y)
x_sample = X[0]
perturb_fn = lambda x: x + np.random.randn(len(x)) * 0.3
stability, exps = compute_explanation_stability(model, x_sample, perturb_fn, n_runs=5)
print(f"Explanation stability: {stability:.3f}")
assert 0 <= stability <= 1, "Stability should be in [0,1]"
print("✓ Explanation stability working")
if __name__ == "__main__":
print("Lab 4: Stability - PASSED")