Generative AI

Generative AI is the branch of AI that produces new content — text, images, audio, code, video — rather than classifying or predicting a label for content that already exists. The distinction matters architecturally: a classifier's output is a fixed-size answer (a label, a score, a box), while a generative model's output is itself a new, often open-ended artifact, which is why generative models need fundamentally different training objectives and sampling procedures than the discriminative models that dominated AI before roughly 2020. At a high level, the major generative approaches split like this:

Autoregressive generation and diffusion solve the same problem — producing a realistic sample from a learned data distribution — in opposite directions, and the choice between them maps directly onto what kind of data is being generated. Autoregressive models (GPT-style LLMs) build output one discrete piece at a time, each new token conditioned on everything generated before it, which fits language's inherently sequential, discrete structure. Diffusion models (Stable Diffusion, Midjourney, Sora) instead start from pure random noise and repeatedly denoise it over dozens of steps toward a coherent image or video frame — a process that fits continuous, high-dimensional pixel data far better than token-by-token generation would, and lets the same denoising framework scale naturally from still images to video.

Generative AI's inference cost structure is the opposite of a classifier's, which has real deployment consequences. A classifier runs once per input and returns a fixed-size answer almost instantly; a generative model's cost scales with how much content it produces — an LLM generating a 2,000-token response runs its forward pass roughly 2,000 times (once per token, unless batched generation or speculative decoding is used), and a diffusion model generating one image runs dozens of full denoising passes through the network. This is the direct reason generative AI inference, not training, now dominates total AI compute spend industry-wide: a chatbot serving millions of users generates tokens continuously, and each token is its own GPU forward pass, at a scale that dwarfs the one-time cost of training the model that's answering.

The hardware response to generative AI's specific cost structure is a real, ongoing shift in chip design, not just more of the same GPUs. NVIDIA's data center roadmap increasingly optimizes for token-generation throughput and memory bandwidth specifically (Hopper and Blackwell's larger HBM capacity, NVLink for multi-GPU inference), because serving generative models at scale is memory-bandwidth-bound in a way training workloads often aren't — the model's weights have to be read from memory for every single token generated, repeatedly, for the life of the deployment, which is a fundamentally different bottleneck than the compute-bound pattern of a one-time training run.

Read generative AI's economics as inverted from classical machine learning's: a classifier's cost is almost entirely upfront in training, while a generative model's cost is ongoing and scales with every token or pixel it ever produces — which is exactly why inference-optimized memory bandwidth, not raw training FLOPs, has become the chip metric the industry is now racing to improve.