Model

A model, in the machine learning sense, is a fixed set of learned numeric parameters — weights and biases arranged in a particular architecture — that maps an input to an output based on patterns extracted from training data, as distinct from the training algorithm that produced it or the raw data it was trained on. A trained model is static until updated: the same weights, the same file, run identically every time it's given the same input, which is what makes a model something you can save to disk, version, benchmark, and deploy as a standalone artifact. At a high level, the pipeline that turns data into a deployable model looks like this:

A model's size is measured in parameter count, and that number maps almost directly onto chip demand. GPT-3 had 175 billion parameters; GPT-4 is estimated in the 1-1.8 trillion range using a mixture-of-experts architecture; Meta's Llama 3.1 topped out at 405 billion parameters released openly. Each parameter is typically stored as a 16-bit or 8-bit number, so a 70-billion-parameter model needs roughly 140GB of memory just to hold its weights at FP16 — before accounting for the activation memory needed during inference. That's why a single model doesn't fit on one GPU past a certain size: it has to be split (sharded) across multiple NVIDIA H100s or B200s connected by NVLink, and HBM capacity per GPU, not raw compute, becomes the hard ceiling on how large a model a given cluster can even load.

Model architecture, not just scale, determines efficiency. A dense model activates every parameter on every input; a mixture-of-experts model like Mixtral or the architecture reportedly behind GPT-4 only routes each token through a fraction of its total parameters (commonly 2 of 8 or 16 experts), which lets a model have a huge total parameter count for knowledge capacity while keeping the active compute per token much lower — a direct trade of memory capacity (still needs all experts loaded in HBM) for inference compute. Distillation goes the other direction: training a smaller "student" model to mimic a larger "teacher" model's outputs, which is how models like DistilBERT or many on-device models get most of a larger model's accuracy at a fraction of the parameter count and memory footprint.

Quantization is the single biggest lever for making a trained model cheaper to run without retraining it. Converting weights from 16-bit floating point down to 8-bit or 4-bit integers roughly halves or quarters memory footprint and often speeds up inference, at a small, usually acceptable accuracy cost — NVIDIA's Hopper and Blackwell architectures added dedicated FP8 and even FP4 tensor core support specifically because the industry standardized on quantized inference as the default for production deployment, not just an edge-device trick.

Read a model not as "the AI" but as a snapshot of weights whose size, architecture, and numeric precision directly set the HBM capacity, GPU count, and interconnect bandwidth required to run it — the same chip economics that decide whether a 60-100-layer wafer yields a usable die also decide, one level up, whether a trillion-parameter model fits in a data center's power and memory budget at all.