Features

Features are the individual measurable input variables a model actually sees — a house-price model's features might be square footage, bedroom count, and zip code; an image model's features are pixel intensities or, for a trained vision network, the learned activation patterns at each layer; a large language model's features are the numeric token embeddings a word gets mapped to before any attention computation happens. A feature is not raw data itself but data converted into a numeric form a model's math can consume, which is why "feature" sits one step downstream of "dataset" and one step upstream of "parameters" in the pipeline. At a high level, raw data becomes usable features like this:

Hand-crafted features used to be most of the work; deep learning moved that work inside the model itself. Classic machine learning (linear regression, decision trees, gradient-boosted trees like XGBoost) relies entirely on a human or a separate pipeline choosing good features — a fraud-detection model's engineers manually craft features like "transactions in the last hour" or "distance from last known location." A convolutional neural network instead learns its own features directly from raw pixels: early layers learn edges and textures, deeper layers learn whole object parts, entirely through backpropagation rather than human feature design — this shift, not a single algorithm, is what "deep learning replaced feature engineering" actually means in practice.

Feature scaling is a small preprocessing step with an outsized effect on training stability. If one feature ranges from 0 to 1 and another ranges from 0 to 1,000,000 (say, normalized age versus raw annual revenue), gradient descent will struggle because the loss surface becomes extremely elongated in one direction — standard practice is to normalize or standardize every feature to a comparable range before training, and skipping this step is one of the most common reasons a model trains slowly or fails to converge at all, independent of architecture or hardware.

Feature dimensionality directly drives compute and memory cost, which is why dimensionality reduction is a real engineering lever, not just a statistics exercise. A naive tabular model with a million one-hot-encoded categorical features costs real GPU memory and FLOPs per training step; techniques like PCA, embeddings (replacing a sparse one-hot vector with a dense 256-dimension learned vector), and feature hashing all exist specifically to compress high-dimensional raw features down to something a GPU can process efficiently without losing the signal that actually predicts the output.

Read a feature as the exact point where real-world data stops being human-readable and starts being a number a chip's matrix-multiply units can act on: whether that conversion happens by hand (classic ML) or inside a trained network's own layers (deep learning), the dimensionality and precision of that final numeric vector is what sets the GPU memory and compute bill for everything downstream — the same conversion-to-silicon discipline that turns a wafer's 60-100+ patterned layers into a working chip.