Explainable AI Interpretability

# Explainable AI & Interpretability

## Introduction & Motivation

XAI: understand model decisions. LIME, SHAP, attention visualization. Applications: model debugging, regulatory compliance.

Motivation: Make black-box models interpretable.

Applications: Medical diagnosis explanation, fairness auditing.

---

## Core Concepts & Theory

### Feature Attribution

Identify important input features.

### Model-Agnostic Methods

Explanation independent of architecture.

### Attention Visualization

Highlight relevant regions.

### Counterfactual Explanations

What-if analysis.

---

## Mathematical Formulation

LIME Local Model:
$$\min_g \sum_i L(f(x_i), g(x_i)) + \Omega(g)$$

SHAP Value:
$$\phi_i = \frac{1}{|S|!|T|!} \sum_{S \subseteq T \setminus \{i\}} |S|!(|T|-|S|-1)![f(S \cup \{i\}) - f(S)]$$

Attention:
$$\alpha_{ij} = \frac{\exp(e_{ij})}{\sum_k \exp(e_{ik})}$$

---

## Advanced Theory & Extensions

### Integrated Gradients

Path-based attribution.

### Layer-wise Relevance Propagation

Decomposition-based explanation.

### Concept Activation Vectors

High-level feature interpretation.

---

## Computational Considerations

LIME: O(K·T) (K=samples, T=training).

SHAP: O(2^|S|) (exponential coalitions).

Attention: O(1) (pre-computed).

---

## Practical Implementation Strategies

### Local Interpretability

Focus on specific predictions.

### Feature Importance Ranking

Order features by contribution.

### Visualization Techniques

Heatmaps and saliency maps.

---

## Benchmark Datasets & Evaluation

ImageNet: Saliency benchmarks.

Medical images: Diagnostic explanation.

COMPAS: Fairness datasets.

---

## Key Challenges & Limitations

### Computational Cost

SHAP exponential complexity.

### Faithfulness

Explanation accuracy verification.

### User Study Validation

Explanation effectiveness assessment.

---

## Hyperparameter Tuning

LIME samples: 1000-10000.

SHAP background: 100-1000.

Threshold: 0.1-0.5.

---

## Real-World Applications & Case Studies

Medical AI: Diagnosis explanation.

Credit Scoring: Decision transparency.

Criminal Justice: Bias detection.

---

## Integration with Other Methods

XAI + bias detection; + model auditing.

---

## Summary & Key Takeaways

Explainable AI makes model decisions interpretable and trustworthy.

Principles:
1. Feature attribution: Importance identification.
2. Model-agnostic: Architecture independence.
3. Local explanation: Instance-specific reasoning.
4. Visualization: Visual interpretation.
5. Faithfulness: Explanation accuracy.

---

## Appendix: Practical Labs

### Lab 1: Feature Importance Ranking

import numpy as np

def compute_feature_importance(model_output, permuted_outputs):
 """Compute feature importance via permutation"""
 baseline = model_output
 importances = baseline - permuted_outputs
 return importances

np.random.seed(42)
baseline = 0.95
permuted = np.array([0.92, 0.93, 0.94, 0.91, 0.89])
importance = compute_feature_importance(baseline, permuted)
print(f"✓ Feature importance: {importance}")

### Lab 2: Attention Visualization

import numpy as np

def visualize_attention(image_features, attention_map, size=224):
 """Create attention-weighted visualization"""
 attention_map = np.resize(attention_map, (size, size))
 attention_map = (attention_map - attention_map.min()) / (attention_map.max() - attention_map.min())
 
 return attention_map

np.random.seed(42)
img = np.random.rand(224, 224, 3)
attn = np.random.rand(14, 14)
vis = visualize_attention(img, attn)
assert vis.shape == attn.shape, "Correct visualization shape"
print("✓ Attention visualization working")

### Lab 3: LIME Sampling

import numpy as np

def lime_sample_neighbors(x, num_samples=1000, kernel_width=0.25):
 """Sample neighbors for LIME local explanation"""
 distances = np.random.normal(0, kernel_width, num_samples)
 x_samples = x + distances
 
 # Exponential kernel weights
 weights = np.exp(-distances**2 / (2 * kernel_width**2))
 return x_samples, weights

np.random.seed(42)
x = np.array([0.5, 0.3, 0.8])
samples, weights = lime_sample_neighbors(x, num_samples=100)
assert samples.shape[0] == 100, "Correct sample count"
assert weights.shape == samples.shape[:1], "Correct weight count"
print("✓ LIME sampling working")

### Lab 4: Saliency Map Generation

import numpy as np

def compute_saliency_map(gradients, x_shape=(224, 224)):
 """Compute saliency map from gradients"""
 saliency = np.max(np.abs(gradients), axis=-1)
 saliency = np.resize(saliency, x_shape)
 saliency = (saliency - saliency.min()) / (saliency.max() - saliency.min())
 
 return saliency

np.random.seed(42)
grads = np.random.randn(14, 14, 3)
saliency = compute_saliency_map(grads)
assert saliency.shape == (224, 224), "Correct saliency shape"
assert np.all(saliency >= 0) and np.all(saliency <= 1), "Normalized saliency"
print("✓ Saliency map generation working")

---