influence functions
**Influence Functions** are a **technique from robust statistics applied to ML that measures how each training example affects a model's prediction** — quantifying the change in a test prediction if a specific training point were upweighted or removed, enabling data attribution and debugging.
**How Influence Functions Work**
- **Question**: How would the model's prediction on test point $z_{test}$ change if training point $z_i$ were removed?
- **Approximation**: $mathcal{I}(z_i, z_{test}) = -
abla_ heta L(z_{test})^T H_{ heta}^{-1}
abla_ heta L(z_i)$ where $H$ is the Hessian.
- **Hessian Inverse**: Computed approximately using conjugate gradients or stochastic estimation.
- **Attribution**: Rank training points by their influence on the test prediction.
**Why It Matters**
- **Data Debugging**: Identify mislabeled, corrupted, or anomalous training examples that hurt predictions.
- **Data Valuation**: Quantify the value or harm of each training data point.
- **Model Debugging**: Understand why a model makes a specific prediction by tracing it to influential training data.
**Influence Functions** are **tracing predictions to training data** — measuring which training examples are most responsible for a model's behavior.