feature extraction

**Feature Extraction** is the **process of using a pre-trained neural network as a fixed feature extractor** — passing input data through the frozen network to obtain learned representations (feature vectors) that can then be used as input to a simpler downstream model. **How Does Feature Extraction Work?** - **Forward Pass**: Run the input through the pre-trained network up to a specific layer. - **Output**: Extract the activation map or feature vector at that layer. - **Downstream**: Feed extracted features into an SVM, logistic regression, k-NN, or MLP. - **Common Layers**: Last hidden layer (global features), intermediate layers (local features), or multi-scale features. **Why It Matters** - **Compute Efficiency**: No backpropagation through the backbone. Features computed once and cached. - **Small Data**: When labeled data is scarce, feature extraction avoids overfitting (fewer trainable parameters). - **Industry**: Many production ML systems use pre-computed embeddings from foundation models. **Feature Extraction** is **treating neural networks as learned feature generators** — leveraging the knowledge encoded in pre-trained models without the cost of end-to-end training.