frozen features

**Frozen Features** refers to **neural network representations that are not updated during training** — the backbone weights are fixed (gradients not computed), and only the downstream task head is trained, preserving the original pre-trained feature space. **What Are Frozen Features?** - **Mechanism**: Set `requires_grad = False` for backbone parameters. Only the classification/regression head has gradients. - **Equivalence**: Linear probing = frozen features + linear head. Feature extraction = frozen features + any downstream model. - **Storage**: Features can be pre-computed and saved to disk for fast downstream experimentation. **Why It Matters** - **Speed**: Orders of magnitude faster training (no backprop through the backbone). - **Memory**: Much lower GPU memory (no need to store intermediate activations for gradient computation). - **Fairness**: Provides a standardized comparison by isolating the quality of the representation from the optimization procedure. **Frozen Features** are **the read-only mode of neural networks** — locking down the learned representations to evaluate their intrinsic quality or enable efficient downstream adaptation.