svcca

**SVCCA** is the **representation comparison method combining singular value decomposition with canonical correlation analysis** - it is used to compare learned subspaces between layers, models, or training checkpoints. **What Is SVCCA?** - **Definition**: SVD reduces noise and dimensionality before CCA measures correlated subspace structure. - **Focus**: Emphasizes shared high-variance representational directions. - **Applications**: Used for studying convergence, transfer, and layer correspondence. - **Output**: Produces correlation scores indicating representational overlap. **Why SVCCA Matters** - **Subspace Insight**: Captures similarity beyond one-to-one neuron alignment assumptions. - **Training Analysis**: Helps identify when representations stabilize during optimization. - **Model Comparison**: Useful for comparing architectures with different parameterizations. - **Interpretability**: Provides structured view of shared representational factors. - **Caveat**: Correlation in subspace does not imply identical causal behavior. **How It Is Used in Practice** - **Dimensional Cut**: Select SVD cutoff carefully to balance noise removal and signal retention. - **Stimulus Robustness**: Repeat analysis on multiple datasets to avoid dataset-specific conclusions. - **Functional Validation**: Pair SVCCA findings with behavioral and intervention tests. SVCCA is **a classical subspace-based method for neural representation comparison** - SVCCA offers useful structural insight when combined with causal and task-level validation.