centered kernel alignment

**Centered kernel alignment** is the **representation similarity metric that compares centered kernel matrices to quantify alignment between activation spaces** - it is widely used for robust layer-to-layer and model-to-model representation comparison. **What Is Centered kernel alignment?** - **Definition**: CKA measures normalized similarity between two feature sets via kernel-based statistics. - **Properties**: Invariant to isotropic scaling and orthogonal transformations in common settings. - **Usage**: Applied to compare layer evolution, transfer learning effects, and training dynamics. - **Variants**: Linear and nonlinear kernels provide different sensitivity profiles. **Why Centered kernel alignment Matters** - **Robust Comparison**: Provides stable similarity scores across models with different widths. - **Training Insight**: Tracks representation drift during fine-tuning and continued pretraining. - **Architecture Study**: Useful for identifying where two models converge or diverge internally. - **Efficiency**: Computationally tractable for many practical interpretability studies. - **Interpretation Limit**: High CKA does not guarantee identical functional circuits. **How It Is Used in Practice** - **Layer Grid**: Compute CKA across full layer pairs to identify correspondence structure. - **Data Consistency**: Use identical stimulus sets and preprocessing for fair comparison. - **Cross-Metric Check**: Validate conclusions with complementary similarity and causal analyses. Centered kernel alignment is **a standard quantitative tool for representation alignment analysis** - centered kernel alignment is strongest when used as part of a broader functional-comparison toolkit.