representer point selection

**Representer Point Selection** is a **data attribution technique that decomposes a model's prediction into a linear combination of training example contributions** — expressing the pre-activation output as $sum_i alpha_i k(x_i, x_{test})$ where $alpha_i$ quantifies training point $i$'s contribution. **How Representer Points Work** - **Representer Theorem**: For L2-regularized models, the pre-activation prediction decomposes into training point contributions. - **Weight $alpha_i$**: $alpha_i = -frac{1}{2lambda n} frac{partial L}{partial f(x_i)}$ — proportional to the gradient of the loss at that training point. - **Kernel**: $k(x_i, x_{test}) = phi(x_i)^T phi(x_{test})$ in the feature space of the penultimate layer. - **Ranking**: Sort training points by $alpha_i cdot k(x_i, x_{test})$ to find the most influential examples. **Why It Matters** - **Decomposition**: Every prediction is explicitly decomposed into training example contributions. - **Proponents/Opponents**: Positive contributions are proponents (support the prediction); negative are opponents. - **Interpretable**: Shows which training examples the model "relies on" for each prediction. **Representer Points** are **predictions explained by training examples** — decomposing every output into specific contributions from individual training data.