<svg viewBox="0 0 760 470" xmlns="http://www.w3.org/2000/svg" font-family="-apple-system,Segoe UI,Roboto,sans-serif">
  <rect x="0" y="0" width="760" height="470" fill="#0d1117"/>
  <text x="380" y="28" fill="#e6edf3" font-size="21" font-weight="700" text-anchor="middle">LLM — How a Language Model Generates Text</text>
  <text x="380" y="48" fill="#8b98a5" font-size="12" text-anchor="middle">predict the next token, one at a time, conditioned on everything before it</text>

  <!-- === AUTOREGRESSIVE GENERATION (the core loop) === -->
  <text x="380" y="72" fill="#e6edf3" font-size="12" font-weight="600" text-anchor="middle">Autoregressive Decoding (one step shown)</text>

  <!-- Input context (tokens so far) -->
  <text x="40" y="92" fill="#6b7684" font-size="9">context (input tokens):</text>
  <g>
    <rect x="40" y="98" width="42" height="20" rx="3" fill="#14261f" stroke="#34d399" stroke-width="0.8"/>
    <text x="61" y="112" fill="#6ee7b7" font-size="8.5" text-anchor="middle">The</text>
    <rect x="86" y="98" width="58" height="20" rx="3" fill="#14261f" stroke="#34d399" stroke-width="0.8"/>
    <text x="115" y="112" fill="#6ee7b7" font-size="8.5" text-anchor="middle">capital</text>
    <rect x="148" y="98" width="30" height="20" rx="3" fill="#14261f" stroke="#34d399" stroke-width="0.8"/>
    <text x="163" y="112" fill="#6ee7b7" font-size="8.5" text-anchor="middle">of</text>
    <rect x="182" y="98" width="55" height="20" rx="3" fill="#14261f" stroke="#34d399" stroke-width="0.8"/>
    <text x="209" y="112" fill="#6ee7b7" font-size="8.5" text-anchor="middle">France</text>
    <rect x="241" y="98" width="30" height="20" rx="3" fill="#14261f" stroke="#34d399" stroke-width="0.8"/>
    <text x="256" y="112" fill="#6ee7b7" font-size="8.5" text-anchor="middle">is</text>
  </g>

  <!-- Arrow into model -->
  <path d="M275,108 L305,108" fill="none" stroke="#8b98a5" stroke-width="1.2"/>
  <polygon points="303,105 309,108 303,111" fill="#8b98a5"/>

  <!-- Transformer stack (the model) -->
  <rect x="312" y="84" width="130" height="50" rx="6" fill="#1c1633" stroke="#a78bfa" stroke-width="1.5"/>
  <!-- Layer lines inside -->
  <g stroke="#3a3a5a" stroke-width="0.5" opacity="0.5">
    <line x1="320" y1="96" x2="434" y2="96"/><line x1="320" y1="104" x2="434" y2="104"/>
    <line x1="320" y1="112" x2="434" y2="112"/><line x1="320" y1="120" x2="434" y2="120"/>
  </g>
  <text x="377" y="102" fill="#c4b5fd" font-size="10" text-anchor="middle" font-weight="600">Transformer</text>
  <text x="377" y="116" fill="#8b98a5" font-size="8" text-anchor="middle">96 layers, 96 heads</text>
  <text x="377" y="128" fill="#6b7684" font-size="7.5" text-anchor="middle">~175B parameters</text>

  <!-- Arrow out to probability -->
  <path d="M442,108 L472,108" fill="none" stroke="#8b98a5" stroke-width="1.2"/>
  <polygon points="470,105 476,108 470,111" fill="#8b98a5"/>

  <!-- Probability distribution (softmax output) -->
  <rect x="480" y="82" width="135" height="56" rx="5" fill="#080d14" stroke="#fbbf24" stroke-width="1.2"/>
  <text x="547" y="97" fill="#fbbf24" font-size="9" text-anchor="middle" font-weight="600">next token probs</text>
  <!-- Bar chart (probability bars) -->
  <rect x="490" y="104" width="80" height="7" rx="1" fill="#fbbf24" opacity="0.8"/>
  <text x="575" y="111" fill="#fbbf24" font-size="7.5">Paris  0.82</text>
  <rect x="490" y="114" width="20" height="7" rx="1" fill="#fbbf24" opacity="0.3"/>
  <text x="515" y="121" fill="#a1701a" font-size="7.5">Lyon   0.04</text>
  <rect x="490" y="124" width="12" height="7" rx="1" fill="#fbbf24" opacity="0.2"/>
  <text x="507" y="131" fill="#a1701a" font-size="7.5">the    0.03</text>

  <!-- Sampled token -->
  <path d="M620,108 L648,108" fill="none" stroke="#fbbf24" stroke-width="1.2"/>
  <polygon points="646,105 652,108 646,111" fill="#fbbf24"/>
  <rect x="655" y="98" width="55" height="22" rx="3" fill="#2a1a0a" stroke="#fbbf24" stroke-width="1.2"/>
  <text x="682" y="113" fill="#fbbf24" font-size="9.5" text-anchor="middle" font-weight="600">Paris</text>

  <!-- Loop arrow back -->
  <path d="M682,120 C682,148 400,148 61,148 C45,148 40,138 40,125" fill="none" stroke="#f87171" stroke-width="1" stroke-dasharray="4,2"/>
  <text x="380" y="158" fill="#f87171" font-size="8.5" text-anchor="middle">append token → repeat (autoregressive loop)</text>

  <!-- === MIDDLE: Architecture overview === -->
  <rect x="30" y="172" width="350" height="150" rx="6" fill="#080d14" stroke="#233043" stroke-width="1.2"/>
  <text x="205" y="190" fill="#e6edf3" font-size="11" text-anchor="middle" font-weight="600">What's Inside (GPT-style decoder-only)</text>

  <!-- Token embedding -->
  <rect x="50" y="200" width="80" height="22" rx="3" fill="#0f2a1f" stroke="#34d399" stroke-width="0.8"/>
  <text x="90" y="214" fill="#6ee7b7" font-size="8" text-anchor="middle">token embed</text>

  <!-- + pos -->
  <text x="140" y="214" fill="#6b7684" font-size="9">+pos</text>

  <!-- Transformer block (repeated) -->
  <rect x="160" y="198" width="140" height="28" rx="3" fill="#1c1633" stroke="#a78bfa" stroke-width="1"/>
  <text x="230" y="210" fill="#c4b5fd" font-size="8.5" text-anchor="middle">Self-Attention → FFN</text>
  <text x="230" y="222" fill="#6b7684" font-size="7" text-anchor="middle">(×N layers)</text>

  <!-- LM head -->
  <rect x="310" y="200" width="55" height="22" rx="3" fill="#2a1a0a" stroke="#f59e0b" stroke-width="0.8"/>
  <text x="337" y="214" fill="#fbbf24" font-size="8" text-anchor="middle">LM head</text>

  <!-- Key components listed -->
  <text x="50" y="248" fill="#8b98a5" font-size="9">Components per block:</text>
  <text x="50" y="264" fill="#8b98a5" font-size="8.5">• Multi-head self-attention (causal mask)</text>
  <text x="50" y="278" fill="#8b98a5" font-size="8.5">• Feed-forward network (4x expansion)</text>
  <text x="50" y="292" fill="#8b98a5" font-size="8.5">• RMSNorm + residual connections</text>
  <text x="50" y="306" fill="#8b98a5" font-size="8.5">• RoPE positional encoding</text>
  <text x="50" y="320" fill="#6b7684" font-size="8">KV cache stores past key/values (memory grows with seq len)</text>

  <!-- === RIGHT: Model landscape === -->
  <rect x="395" y="172" width="335" height="150" rx="6" fill="#0b1220" stroke="#233043" stroke-width="1"/>
  <text x="562" y="190" fill="#e6edf3" font-size="11" text-anchor="middle" font-weight="600">Frontier LLMs (2024-2025)</text>

  <text x="415" y="210" fill="#76b900" font-size="9.5" font-weight="600">GPT-4o (OpenAI)</text>
  <text x="540" y="210" fill="#8b98a5" font-size="8.5">~1.8T params, MoE</text>

  <text x="415" y="228" fill="#e0913a" font-size="9.5" font-weight="600">Claude 3.5 (Anthropic)</text>
  <text x="560" y="228" fill="#8b98a5" font-size="8.5">200K context</text>

  <text x="415" y="246" fill="#60a5fa" font-size="9.5" font-weight="600">Gemini 1.5 (Google)</text>
  <text x="555" y="246" fill="#8b98a5" font-size="8.5">1M token context</text>

  <text x="415" y="264" fill="#c4b5fd" font-size="9.5" font-weight="600">Llama-3.1 405B (Meta)</text>
  <text x="575" y="264" fill="#8b98a5" font-size="8.5">open weights</text>

  <text x="415" y="282" fill="#fbbf24" font-size="9.5" font-weight="600">DeepSeek-V3 (DeepSeek)</text>
  <text x="575" y="282" fill="#8b98a5" font-size="8.5">671B MoE, 37B active</text>

  <text x="415" y="302" fill="#34d399" font-size="9.5" font-weight="600">Qwen-2.5 72B (Alibaba)</text>
  <text x="575" y="302" fill="#8b98a5" font-size="8.5">multilingual</text>

  <text x="562" y="320" fill="#6b7684" font-size="8" text-anchor="middle">All decoder-only transformers. All trained on 10-15T tokens.</text>

  <!-- === BOTTOM: Training + inference cost === -->
  <rect x="30" y="335" width="700" height="80" rx="5" fill="#0b1220" stroke="#233043" stroke-width="1"/>
  <text x="380" y="354" fill="#e6edf3" font-size="10" text-anchor="middle" font-weight="600">The Economics</text>

  <text x="50" y="374" fill="#f87171" font-size="9.5">Training: 100M+ USD, 10K-100K GPUs, months</text>
  <text x="50" y="390" fill="#8b98a5" font-size="9">GPT-4: ~25K A100s for 90 days = ~100M USD compute</text>

  <text x="420" y="374" fill="#34d399" font-size="9.5">Inference: cost/token matters at scale</text>
  <text x="420" y="390" fill="#8b98a5" font-size="9">~1B queries/day → serving costs > training costs</text>

  <text x="380" y="408" fill="#6b7684" font-size="9" text-anchor="middle">Training is memory-bound (large models). Inference is latency-bound (time-to-first-token + tokens/sec).</text>

  <!-- Footer -->
  <text x="380" y="435" fill="#e6edf3" font-size="9.5" text-anchor="middle" font-weight="600">An LLM is a giant lookup table of language compressed into neural weights — next-token prediction is all it does.</text>

  <text x="380" y="460" fill="#6b7684" font-size="11" text-anchor="middle">Predict the next word, at scale, trained on the internet. That's it. That's the revolution.</text>
</svg>
```large language model (LLM)** is a neural network with billions of parameters, trained on internet-scale text to do one deceptively simple thing: predict the next token given the tokens so far. Scaled up far enough, that single objective produces systems that write fluent prose, answer questions, generate working code, translate languages, and follow instructions — capabilities nobody explicitly programmed in. GPT, Claude, Llama, and Gemini are all LLMs. The diagram traces what actually happens between a prompt going in and a word coming out.\n\n```svg\n<svg viewBox="0 0 760 560" xmlns="http://www.w3.org/2000/svg" font-family="-apple-system,Segoe UI,Roboto,sans-serif">\n  <rect x="0" y="0" width="760" height="560" fill="#0d1117"/>\n  <text x="380" y="34" fill="#e6edf3" font-size="21" font-weight="700" text-anchor="middle">Anatomy of a Large Language Model</text>\n  <text x="380" y="56" fill="#8b98a5" font-size="12.5" text-anchor="middle">text in, one token out — repeated until the answer is complete</text>\n  <rect x="140.0" y="96" width="220" height="44" rx="6" fill="#1c1633" stroke="#a78bfa" stroke-width="1.4"/><text x="250" y="114" fill="#c4b5fd" font-size="13" font-weight="700" text-anchor="middle">"Write me a poem"</text><text x="250" y="131" fill="#9aa4b2" font-size="12" text-anchor="middle">raw text prompt</text>\n  <path d="M250 140 L250 156 M245 148 L250 156 L255 148" stroke="#5b6b7f" stroke-width="1.3" fill="none"/>\n  <rect x="140.0" y="156" width="220" height="44" rx="6" fill="#0e2f4a" stroke="#38bdf8" stroke-width="1.4"/><text x="250" y="174" fill="#7dd3fc" font-size="13" font-weight="700" text-anchor="middle">Tokenizer</text><text x="250" y="191" fill="#9aa4b2" font-size="12" text-anchor="middle">text → integer token IDs</text>\n  <path d="M250 200 L250 216 M245 208 L250 216 L255 208" stroke="#5b6b7f" stroke-width="1.3" fill="none"/>\n  <rect x="140.0" y="216" width="220" height="44" rx="6" fill="#134e3a" stroke="#34d399" stroke-width="1.4"/><text x="250" y="234" fill="#6ee7b7" font-size="13" font-weight="700" text-anchor="middle">Embeddings + position</text><text x="250" y="251" fill="#9aa4b2" font-size="12" text-anchor="middle">IDs → vectors the model reads</text>\n  <path d="M250 260 L250 276 M245 268 L250 276 L255 268" stroke="#5b6b7f" stroke-width="1.3" fill="none"/>\n  <rect x="132.0" y="270" width="236" height="150" rx="9" fill="none" stroke="#3f5169" stroke-width="1.2" stroke-dasharray="5 4"/>\n  <text x="362.0" y="348" fill="#8b98a5" font-size="13" transform="rotate(90 376.0 348)"> × N layers</text>\n  <rect x="140.0" y="276" width="220" height="44" rx="6" fill="#3b2f0a" stroke="#fbbf24" stroke-width="1.4"/><text x="250" y="294" fill="#fcd34d" font-size="13" font-weight="700" text-anchor="middle">Self-attention</text><text x="250" y="311" fill="#9aa4b2" font-size="12" text-anchor="middle">tokens exchange information</text>\n  <path d="M250 320 L250 334 M245 326 L250 334 L255 326" stroke="#5b6b7f" stroke-width="1.3" fill="none"/>\n  <rect x="140.0" y="334" width="220" height="44" rx="6" fill="#3b2f0a" stroke="#fbbf24" stroke-width="1.4"/><text x="250" y="352" fill="#fcd34d" font-size="13" font-weight="700" text-anchor="middle">Feed-forward (MLP)</text><text x="250" y="369" fill="#9aa4b2" font-size="12" text-anchor="middle">each token processed alone</text>\n  <path d="M250 420 L250 436 M245 428 L250 436 L255 428" stroke="#5b6b7f" stroke-width="1.3" fill="none"/>\n  <rect x="140.0" y="436" width="220" height="44" rx="6" fill="#0e2f4a" stroke="#38bdf8" stroke-width="1.4"/><text x="250" y="454" fill="#7dd3fc" font-size="13" font-weight="700" text-anchor="middle">Unembed → logits</text><text x="250" y="471" fill="#9aa4b2" font-size="12" text-anchor="middle">vector → score per token</text>\n  <path d="M250 480 L250 496 M245 488 L250 496 L255 488" stroke="#5b6b7f" stroke-width="1.3" fill="none"/>\n  <rect x="140.0" y="496" width="220" height="44" rx="6" fill="#1e2530" stroke="#f87171" stroke-width="1.4"/><text x="250" y="514" fill="#fca5a5" font-size="13" font-weight="700" text-anchor="middle">Next-token probabilities</text><text x="250" y="531" fill="#9aa4b2" font-size="12" text-anchor="middle">sample one → append → repeat</text>\n  <path d="M360.0 518 C510.0 518 510.0 118 360.0 118" fill="none" stroke="#f87171" stroke-width="1.5" stroke-dasharray="5 4"/>\n  <path d="M368.0 113 L360.0 118 L368.0 123" fill="none" stroke="#f87171" stroke-width="1.5"/>\n  <text x="455.0" y="318" fill="#fca5a5" font-size="12" text-anchor="middle">autoregressive</text>\n  <text x="455.0" y="331" fill="#fca5a5" font-size="12" text-anchor="middle">loop</text>\n  <text x="560" y="150" fill="#e6edf3" font-size="13" font-weight="700">Scale</text>\n  <text x="560" y="176" fill="#c4b5fd" font-size="13" font-weight="700">Parameters</text>\n  <text x="560" y="191" fill="#8b98a5" font-size="12">10^9 – 10^12 weights</text>\n  <text x="560" y="218" fill="#7dd3fc" font-size="13" font-weight="700">Training tokens</text>\n  <text x="560" y="233" fill="#8b98a5" font-size="12">trillions of words</text>\n  <text x="560" y="260" fill="#6ee7b7" font-size="13" font-weight="700">Context window</text>\n  <text x="560" y="275" fill="#8b98a5" font-size="12">thousands – millions</text>\n  <text x="560" y="302" fill="#fcd34d" font-size="13" font-weight="700">Objective</text>\n  <text x="560" y="317" fill="#8b98a5" font-size="12">predict the next token</text>\n  <text x="560" y="344" fill="#fca5a5" font-size="13" font-weight="700">Emerges</text>\n  <text x="560" y="359" fill="#8b98a5" font-size="12">language, reasoning, code</text>\n  <text x="380" y="548" fill="#6b7684" font-size="13" text-anchor="middle">The whole model does one thing — predict the next token — and everything else is an emergent consequence of doing it at scale.</text>\n</svg>\n```\n\n**Everything is next-token prediction.** During training the model sees enormous amounts of text with the next word hidden, and it adjusts its weights to raise the probability it would have assigned to the real next token. There is no separate "reasoning module" or "fact database" — grammar, world knowledge, translation, and arithmetic are all compressed into the weights as a side effect of getting good at this one guessing game.\n\n**The transformer block is the repeating unit.** Each layer has two parts: a self-attention step, where every token looks at the others and pulls in the context it needs, and a feed-forward network that processes each position independently. Stacking dozens to over a hundred of these blocks lets early layers capture surface patterns and later layers capture meaning, syntax, and long-range structure.\n\n**Scale is the defining property.** LLMs are distinguished from earlier language models by sheer size — parameters, training tokens, and compute. Empirical scaling laws show loss falling predictably as all three grow together, and certain abilities (in-context learning, multi-step reasoning) appear only past a size threshold. This predictability is why labs are willing to spend enormous sums on a single training run.\n\n**Pretraining teaches language; post-training teaches behavior.** A raw pretrained model is a talented autocomplete engine but not yet a helpful assistant. A second stage — instruction tuning on curated examples, then reinforcement learning from human feedback (RLHF) — aligns it to follow instructions, stay on task, and refuse harmful requests. Most of the "personality" of a deployed chatbot comes from this phase, not pretraining.\n\n**Inference is autoregressive.** To answer, the model generates one token, appends it to the input, and runs again — looping until it emits a stop token. Each step reuses cached attention state (the KV cache) so it does not recompute the whole history, which is why the first token is slow (prefill) and later tokens are fast (decode).\n\n| Component | Role | Analogy |\n|---|---|---|\n| Tokenizer | splits text into subword tokens | breaking a sentence into Lego pieces |\n| Embeddings | turn token IDs into vectors | giving each piece coordinates in meaning-space |\n| Attention | tokens share context | everyone in the room comparing notes |\n| Feed-forward | per-token processing | each token thinking on its own |\n| Unembedding | vectors back to token scores | scoring every possible next word |\n\nRead an LLM through a *next-token-prediction* lens rather than a *knowledge-database* lens: it does not look facts up, it reconstructs the most probable continuation from patterns compressed into its weights during training. That single framing explains its strengths — fluency, generalization, in-context learning — and its failure modes — confident hallucination, sensitivity to phrasing, and knowledge frozen at its training cutoff — because all of them fall out of a system optimized to predict text rather than to store truth.\n