shap

**SHAP (SHapley Additive exPlanations)** is the **game-theoretic framework for explaining machine learning model predictions by computing each feature's fair marginal contribution to the prediction** — derived from Shapley values in cooperative game theory, providing a unified, theoretically grounded explanation method applicable to any ML model. **What Is SHAP?** - **Definition**: A method that explains individual model predictions by assigning each input feature a Shapley value — the average marginal contribution of that feature across all possible subsets of features, measuring how much the feature shifted the prediction from the expected baseline. - **Foundation**: Shapley values from cooperative game theory (Lloyd Shapley, Nobel Prize in Economics 2012) — a mathematically unique method for fairly attributing a cooperative outcome among players based on their marginal contributions. - **Analogy**: Treat each feature as a "player" in a cooperative game where the "payout" is the model prediction. SHAP fairly divides credit: "Your credit score of 750 increased loan approval probability by +0.12; income of $80k added +0.08; late payment history subtracted -0.15." - **Publication**: "A Unified Approach to Interpreting Model Predictions" — Lundberg & Lee, UW (2017). **Why SHAP Matters** - **Theoretical Soundness**: The only additive feature attribution method satisfying three mathematically proven axioms: Local Accuracy (attributions sum to prediction), Missingness (absent features get zero attribution), and Consistency (more impactful features always get higher values). - **Model-Agnostic**: Works for any model — linear regression, gradient boosting, neural networks, random forests — with different computational approaches optimized for each. - **Consistent Across Methods**: SHAP unifies many prior methods (LIME, DeepLIFT, LRP) — showing they are all approximations of Shapley values, providing theoretical grounding for their empirical successes. - **Global + Local Explanations**: Individual Shapley values explain specific predictions; aggregating across the dataset provides global feature importance with consistent interpretability. - **Industry Standard**: Deployed widely in finance (credit scoring explanation), healthcare (clinical risk model explanation), and ML platforms (Azure ML, AWS SageMaker, Google Vertex AI). **SHAP Computation Methods** **KernelSHAP (Model-Agnostic, Slow)**: - Approximate Shapley values by training a weighted linear model on all feature subsets. - Theoretically exact in the limit; approximation quality depends on number of samples. - Works for any model; slow for high-dimensional inputs (many features). **TreeSHAP (Tree Models, Fast)**: - Exact Shapley values in polynomial time O(TLD²) for tree-based models (decision trees, random forests, XGBoost, LightGBM). - Native support in XGBoost, LightGBM, CatBoost. - Orders of magnitude faster than KernelSHAP for tree models. **DeepSHAP (Neural Networks)**: - Combines DeepLIFT backpropagation with Shapley value theory. - Approximate but fast for deep neural networks. - Satisfies SHAP axioms approximately. **GradientSHAP**: - Combines Integrated Gradients with SHAP — samples from a distribution of baselines, averages gradients. - Better baseline handling than single-baseline Integrated Gradients. **SHAP Visualizations** **Force Plot**: - Shows how each feature's Shapley value pushes the prediction above or below the baseline. - Red features increase prediction; blue features decrease. - Stacked horizontally to show the complete "force" driving the output. **Summary Plot (Beeswarm)**: - Each dot is one sample; x-position is Shapley value; color is feature value. - Shows distribution of feature impacts across dataset. - Most informative global visualization for understanding feature behavior. **Dependence Plot**: - Plot SHAP value vs. feature value for one feature. - Reveals non-linear relationships and interaction effects. **Waterfall Plot**: - Step-by-step breakdown of a single prediction — shows exactly how each feature moved the prediction from baseline. **Shapley Value Properties** | Property | Guarantee | Practical Meaning | |----------|-----------|-------------------| | Efficiency | Σ φ_i = f(x) - E[f(x)] | Attributions sum to prediction - baseline | | Symmetry | Equal contribution → equal value | Fair treatment of correlated features | | Dummy | Zero contribution → zero value | Irrelevant features get no credit | | Additivity | Combined models → summed values | Consistent across model ensembles | **SHAP in Regulated Industries** - **Credit**: Explain why a loan was denied in terms of specific contributing features — complying with adverse action notice requirements (ECOA, FCRA). - **Healthcare**: Show clinicians which vital signs and lab values drove a sepsis risk score — enabling clinical validation. - **Insurance**: Explain premium calculations in terms of risk factors — required by insurance regulators in many jurisdictions. **SHAP Limitations** - **Computational Cost**: KernelSHAP requires exponentially many model evaluations; TreeSHAP is fast only for trees. - **Correlation Handling**: Shapley values assume feature independence for subset sampling — correlated features can produce counter-intuitive attributions. - **Not Causal**: SHAP explains model behavior, not causal relationships — high SHAP value for a feature doesn't mean changing that feature will change the outcome in the real world. SHAP is **the unified theory of feature attribution that gave machine learning explainability a mathematical foundation** — by grounding explanations in 70 years of cooperative game theory, SHAP provides the principled, consistent, and auditable explanations that high-stakes AI deployment demands across every regulated industry.