large language models

What is it?

Large language models (LLMs) are deep learning models — almost universally built on the transformer architecture — trained on large text (and increasingly multimodal) corpora to predict and generate sequences of tokens, the sub-word units models actually operate on.

How does it work?

A transformer represents each token as an embedding — a vector capturing learned semantic and positional information — and processes sequences through stacked self-attention layers, where attention lets the model weigh how relevant every other token in the context is to the one currently being processed, regardless of distance between them. A base model is pretrained on broad data, then commonly fine-tuned on narrower, task- or domain-specific data to specialize its behavior. Because a model's knowledge is fixed at training time and its context window is finite, retrieval-augmented generation (RAG) is widely used to supply a model with current or domain-specific information at inference time by retrieving relevant documents and including them in the prompt, rather than relying solely on parametric memory. Evaluation measures how well a model performs on defined tasks and benchmarks, and safety work addresses failure modes — hallucination, bias, unwanted or harmful outputs — that evaluation on capability alone does not catch.

Why does it matter?

LLMs are the technology most directly responsible for the current surge in demand for the AI accelerator hardware described in the previous pillar, creating direct, practical linkage between semiconductor design/manufacturing capacity and LLM capability and cost.

How does it connect to the next layer?

Understanding how LLMs work is the prerequisite for the next pillar's subject: where semiconductor and AI knowledge — including LLMs specifically — gets put to work on real problems, covered in Applications.