dimensionality reduction for embeddings

Dimensionality reduction compresses high-dimensional embeddings to lower dimensions while preserving similarity structure. **Why reduce**: Lower storage costs, faster similarity search, reduce noise, enable visualization. **Methods**: **PCA**: Linear projection to principal components. Fast, effective for linear structure. **UMAP**: Preserves local and global structure. Good for visualization. **t-SNE**: Preserves local structure. Primarily for 2D/3D visualization. **Autoencoders**: Learn nonlinear compression. Can be fine-tuned. **Random projection**: Fast, simple, works via Johnson-Lindenstrauss lemma. **Trade-offs**: Information loss, reconstruction error, changed similarity rankings. Validate that downstream task performance is acceptable. **Typical reductions**: 1536-dim to 256-dim or 512-dim common. Aggressive reduction (to 64) may hurt quality. **For vector search**: Smaller vectors = faster search, less memory. But may need to evaluate more candidates to maintain recall. **Training on data**: PCA, autoencoders need to be fit on representative data. Matryoshka embeddings provide built-in reduction. **Evaluation**: Compare retrieval quality at different dimensions. Find acceptable trade-off point.