explainable ai
**Explainable AI develops representations and methods that help people understand, inspect, contest, and act on AI-system behavior.** Explanations support debugging, scientific insight, model validation, regulated or high-impact decisions, operator trust calibration, failure analysis, and user recourse, but can mislead when they are unstable or unfaithful. Interpretability may be intrinsic to a transparent model or post-hoc for a complex model; explanations may be global or local, feature-based, example-based, concept-based, counterfactual, causal, mechanistic, or natural-language. Audience and decision determine the useful form. A professional responsible-AI claim identifies affected people, intended benefit, prohibited use, decision authority, data provenance, model capability, foreseeable misuse, uncertainty, recourse, monitoring, and accountable owner. Fairness, privacy, transparency, safety, accessibility, autonomy, and reliability can conflict and require explicit tradeoffs rather than a single ethics score.
**Architecture, representation, and operating mechanism.** SHAP attributes a prediction using Shapley-inspired values under a background/feature-dependence choice; LIME fits a local surrogate around perturbed samples; gradients and integrated gradients attribute differentiable outputs; saliency visualizes inputs; concept activation probes human concepts; counterfactuals search actionable changes. An explanation method receives model, input, output, reference distribution or perturbation process, and constraints, then produces attributions, examples, rules, concepts, or alternatives. A user interface communicates scope and uncertainty, and feedback/testing checks whether the explanation supports the intended task. Fidelity to model behavior, stability, sensitivity, completeness, localization, sparsity, plausibility, actionability, computational cost, human comprehension, decision improvement, trust calibration, subgroup consistency, and resistance to manipulation matter. Interfaces, defaults, incentives, human workflow, automation level, tool permissions, business policy, organizational governance, and downstream action often determine harm more than the model score. Defense in depth limits consequence when predictions are wrong or misused. Evaluation combines task utility with subgroup and intersectional performance, calibration, harmful-error severity, robustness, privacy risk, explanation fidelity, human override, complaint and appeal outcomes, incident rate, latency, cost, and uncertainty. Aggregate accuracy can conceal systematic harm, and a fairness metric chosen after seeing results can rationalize rather than govern.
**Implementation, infrastructure, and failure modes.** Tree/rule models offer intrinsic structure; local surrogate sampling must respect data manifold; SHAP approximations choose explainers/backgrounds; attention visualization is not automatically causal explanation; concept probes require validated concepts; mechanistic interpretability traces circuits/features in networks. Post-hoc methods may require many forward/backward evaluations and large activation capture, stressing GPUs and storage. Efficient batching, sampling, low-rank probes, activation caches, on-device summaries, and privacy-aware logging shape deployability. Saliency changes under tiny perturbations, correlated features make attribution ambiguous, explanations are cherry-picked, natural-language rationales are plausible but unfaithful, attention weights are overclaimed, counterfactuals are infeasible, and users become overconfident. Engineering includes data movement, finite precision, concurrency, resource contention, security boundaries, error propagation, and deterministic behavior when assumptions fail. Problem selection, impact assessment, collection, consent or lawful basis, labeling, training, evaluation, deployment, monitoring, feedback, incident response, update, retention, deletion, and retirement form one lifecycle. Decisions, datasets, model cards, approvals, exceptions, and user communications remain traceable.
**Evaluation, governance, and deployment.** Use sanity checks with randomized model/labels, deletion/insertion or retraining tests, repeated seeds/backgrounds, correlated-feature stress, adversarial explanation manipulation, domain-expert review, user studies measuring decisions, and comparison with known synthetic ground truth. Prediction, explanation service, data provenance, model/version, confidence, policy, user interface, human review, appeal, audit log, and corrective action form the workflow. Explanation does not replace accuracy, fairness, privacy, or accountability. High-impact uses document explanation purpose, audience, limitations, trade secrets, privacy, accessibility, retention, contestability, and who can override. Legal requirements vary and should not be reduced to a generic right-to-explanation slogan. Assurance combines documentation, data and label audits, red teaming, robustness and privacy tests, subgroup evaluation, causal or counterfactual analysis where appropriate, human-factors studies, accessibility testing, external review, incident exercises, and post-deployment monitoring. Technical tests do not replace legal, domain, or community judgment. Problem selection, impact assessment, collection, consent or lawful basis, labeling, training, evaluation, deployment, monitoring, feedback, incident response, update, retention, deletion, and retirement form one lifecycle. Decisions, datasets, model cards, approvals, exceptions, and user communications remain traceable. Evaluation combines task utility with subgroup and intersectional performance, calibration, harmful-error severity, robustness, privacy risk, explanation fidelity, human override, complaint and appeal outcomes, incident rate, latency, cost, and uncertainty. Aggregate accuracy can conceal systematic harm, and a fairness metric chosen after seeing results can rationalize rather than govern.
| Method | Scope | Output | Strength | Limitation |
|---|---|---|---|---|
| SHAP | Local aggregated/global | Feature attributions | Consistent additive framework | Background/dependence/cost |
| LIME | Local | Sparse surrogate weights | Model-agnostic/simple | Sampling instability/fidelity |
| Gradient methods | Local differentiable | Input attribution map | Fast model-aware | Saturation/noise |
| Attention visualization | Internal/local | Attention patterns | Easy Transformer inspection | Not causal proof |
| Concept/counterfactual | Concept or actionable local | Concept score/alternative | Human-oriented | Concept validity/feasibility |
```svg
```
**Selection and practical application.** Use intrinsic models when transparency and performance permit, SHAP for structured attribution with assumptions stated, LIME for exploratory local surrogates, gradients for differentiable models, counterfactuals for actionable options, and concepts/mechanistic methods for deeper analysis. Credit and risk review, medicine, industrial diagnostics, fraud, model debugging, scientific ML, autonomous operations, content moderation, and foundation-model analysis use explanations. Interfaces, defaults, incentives, human workflow, automation level, tool permissions, business policy, organizational governance, and downstream action often determine harm more than the model score. Defense in depth limits consequence when predictions are wrong or misused. A professional responsible-AI claim identifies affected people, intended benefit, prohibited use, decision authority, data provenance, model capability, foreseeable misuse, uncertainty, recourse, monitoring, and accountable owner. Fairness, privacy, transparency, safety, accessibility, autonomy, and reliability can conflict and require explicit tradeoffs rather than a single ethics score. CFS connects this topic to semiconductor architecture, implementation, verification, manufacturing, packaging, test, and deployed AI-system tradeoffs across the platform.