<svg viewBox="0 0 760 470" xmlns="http://www.w3.org/2000/svg" font-family="-apple-system,Segoe UI,Roboto,sans-serif">
  <rect x="0" y="0" width="760" height="470" fill="#0d1117"/>
  <text x="380" y="28" fill="#e6edf3" font-size="21" font-weight="700" text-anchor="middle">Vision Transformer (ViT) — Patches Are All You Need</text>
  <text x="380" y="48" fill="#8b98a5" font-size="12" text-anchor="middle">split image into patches, flatten to tokens, add position embeddings, pass through standard transformer encoder</text>

  <!-- === TOP: ViT architecture === -->
  <rect x="25" y="62" width="710" height="148" rx="6" fill="#080d14" stroke="#233043" stroke-width="1.2"/>
  <text x="380" y="82" fill="#e6edf3" font-size="11" text-anchor="middle" font-weight="600">ViT Pipeline: Image → Patches → Tokens → Transformer → Classification</text>

  <!-- Image -->
  <rect x="40" y="95" width="55" height="55" rx="3" fill="#14261f" stroke="#34d399" stroke-width="0.9"/>
  <text x="67" y="126" fill="#6ee7b7" font-size="8" text-anchor="middle">224×224</text>

  <!-- Grid lines showing patches -->
  <line x1="40" y1="113" x2="95" y2="113" stroke="#34d399" stroke-width="0.3"/>
  <line x1="40" y1="131" x2="95" y2="131" stroke="#34d399" stroke-width="0.3"/>
  <line x1="58" y1="95" x2="58" y2="150" stroke="#34d399" stroke-width="0.3"/>
  <line x1="76" y1="95" x2="76" y2="150" stroke="#34d399" stroke-width="0.3"/>

  <path d="M98,122 L118,122" fill="none" stroke="#8b98a5" stroke-width="0.7"/>
  <polygon points="116,119 122,122 116,125" fill="#8b98a5"/>

  <!-- Patches flattened -->
  <rect x="125" y="95" width="22" height="55" rx="2" fill="#0f1a2a" stroke="#60a5fa" stroke-width="0.7"/>
  <rect x="150" y="95" width="22" height="55" rx="2" fill="#0f1a2a" stroke="#60a5fa" stroke-width="0.7"/>
  <rect x="175" y="95" width="22" height="55" rx="2" fill="#0f1a2a" stroke="#60a5fa" stroke-width="0.7"/>
  <text x="160" y="165" fill="#93c5fd" font-size="7.5" text-anchor="middle">N = (224/16)² = 196 patches</text>

  <path d="M200,122 L218,122" fill="none" stroke="#8b98a5" stroke-width="0.7"/>
  <polygon points="216,119 222,122 216,125" fill="#8b98a5"/>

  <!-- Linear projection + position embed -->
  <rect x="225" y="92" width="90" height="62" rx="4" fill="#2a1a0a" stroke="#f59e0b" stroke-width="0.9"/>
  <text x="270" y="112" fill="#fbbf24" font-size="8" text-anchor="middle">Linear proj</text>
  <text x="270" y="126" fill="#fbbf24" font-size="8" text-anchor="middle">+ pos embed</text>
  <text x="270" y="140" fill="#6b7684" font-size="7" text-anchor="middle">+ [CLS] token</text>
  <text x="270" y="165" fill="#a1701a" font-size="7" text-anchor="middle">16×16×3 → d=768</text>

  <path d="M318,122 L338,122" fill="none" stroke="#8b98a5" stroke-width="0.7"/>
  <polygon points="336,119 342,122 336,125" fill="#8b98a5"/>

  <!-- Transformer encoder (L layers) -->
  <rect x="345" y="88" width="180" height="70" rx="6" fill="#1a1520" stroke="#a78bfa" stroke-width="1.2"/>
  <text x="435" y="110" fill="#c4b5fd" font-size="10" text-anchor="middle" font-weight="600">Transformer Encoder</text>
  <text x="435" y="126" fill="#8b98a5" font-size="8" text-anchor="middle">L=12 layers, h=12 heads</text>
  <text x="435" y="140" fill="#8b98a5" font-size="8" text-anchor="middle">MHSA + FFN + LayerNorm</text>
  <text x="435" y="154" fill="#6b7684" font-size="7" text-anchor="middle">all 196+1 tokens attend to all others</text>

  <path d="M528,122 L548,122" fill="none" stroke="#8b98a5" stroke-width="0.7"/>
  <polygon points="546,119 552,122 546,125" fill="#8b98a5"/>

  <!-- CLS head -->
  <rect x="555" y="100" width="80" height="44" rx="4" fill="#14261f" stroke="#34d399" stroke-width="0.9"/>
  <text x="595" y="120" fill="#6ee7b7" font-size="8.5" text-anchor="middle">[CLS] → MLP</text>
  <text x="595" y="135" fill="#6b7684" font-size="7.5" text-anchor="middle">→ class logits</text>

  <!-- Output -->
  <path d="M638,122 L658,122" fill="none" stroke="#8b98a5" stroke-width="0.7"/>
  <rect x="661" y="108" width="55" height="28" rx="3" fill="#14261f" stroke="#34d399" stroke-width="0.8"/>
  <text x="688" y="125" fill="#6ee7b7" font-size="8" text-anchor="middle">cat: 0.97</text>

  <!-- Key difference callout -->
  <text x="380" y="195" fill="#fbbf24" font-size="8.5" text-anchor="middle">No convolutions, no pooling — pure self-attention on patch tokens. Needs large-scale pretraining (ImageNet-21K or JFT).</text>

  <!-- === MIDDLE LEFT: ViT variants === -->
  <rect x="25" y="215" width="350" height="112" rx="6" fill="#0b1220" stroke="#233043" stroke-width="1"/>
  <text x="200" y="233" fill="#e6edf3" font-size="10" text-anchor="middle" font-weight="600">ViT Family</text>

  <text x="45" y="253" fill="#60a5fa" font-size="8.5" font-weight="600">ViT-B/16:</text>
  <text x="110" y="253" fill="#8b98a5" font-size="8.5">86M params, patch=16, d=768 (original)</text>
  <text x="45" y="269" fill="#34d399" font-size="8.5" font-weight="600">ViT-L/14:</text>
  <text x="110" y="269" fill="#8b98a5" font-size="8.5">307M, CLIP backbone (standard VLM encoder)</text>
  <text x="45" y="285" fill="#fbbf24" font-size="8.5" font-weight="600">DINOv2:</text>
  <text x="100" y="285" fill="#8b98a5" font-size="8.5">self-supervised ViT (no labels, Meta)</text>
  <text x="45" y="301" fill="#c4b5fd" font-size="8.5" font-weight="600">SigLIP:</text>
  <text x="95" y="301" fill="#8b98a5" font-size="8.5">sigmoid loss CLIP (better scaling, Google)</text>
  <text x="45" y="317" fill="#f87171" font-size="8.5" font-weight="600">EVA-02:</text>
  <text x="100" y="317" fill="#8b98a5" font-size="8.5">masked image modeling + CLIP (best open)</text>

  <!-- === MIDDLE RIGHT: ViT vs CNN === -->
  <rect x="390" y="215" width="345" height="112" rx="6" fill="#0b1220" stroke="#233043" stroke-width="1"/>
  <text x="562" y="233" fill="#e6edf3" font-size="10" text-anchor="middle" font-weight="600">ViT vs CNN</text>

  <text x="410" y="255" fill="#34d399" font-size="8.5" font-weight="600">ViT advantages:</text>
  <text x="410" y="271" fill="#8b98a5" font-size="8.5">• Global attention from layer 1 (no receptive field limit)</text>
  <text x="410" y="287" fill="#8b98a5" font-size="8.5">• Scales better with data/compute (scaling laws)</text>
  <text x="410" y="303" fill="#8b98a5" font-size="8.5">• Same architecture as text → easy multimodal fusion</text>

  <text x="410" y="321" fill="#f87171" font-size="8.5" font-weight="600">ViT limitations:</text>
  <text x="560" y="321" fill="#8b98a5" font-size="8.5">needs more data, O(n²) in patches</text>

  <!-- === BOTTOM: Where ViT lives === -->
  <rect x="25" y="337" width="710" height="65" rx="5" fill="#0b1220" stroke="#233043" stroke-width="1"/>
  <text x="380" y="355" fill="#e6edf3" font-size="10" text-anchor="middle" font-weight="600">Where ViTs Are Used (2024+)</text>

  <text x="95" y="377" fill="#c4b5fd" font-size="9" text-anchor="middle" font-weight="600">VLM encoders</text>
  <text x="95" y="391" fill="#8b98a5" font-size="8" text-anchor="middle">GPT-4V, Gemini, LLaVA</text>

  <text x="245" y="377" fill="#fbbf24" font-size="9" text-anchor="middle" font-weight="600">CLIP / retrieval</text>
  <text x="245" y="391" fill="#8b98a5" font-size="8" text-anchor="middle">image↔text search</text>

  <text x="395" y="377" fill="#34d399" font-size="9" text-anchor="middle" font-weight="600">Diffusion backbone</text>
  <text x="395" y="391" fill="#8b98a5" font-size="8" text-anchor="middle">DiT (Flux, Sora)</text>

  <text x="545" y="377" fill="#60a5fa" font-size="9" text-anchor="middle" font-weight="600">Medical imaging</text>
  <text x="545" y="391" fill="#8b98a5" font-size="8" text-anchor="middle">pathology, radiology</text>

  <text x="680" y="377" fill="#f87171" font-size="9" text-anchor="middle" font-weight="600">Self-driving</text>
  <text x="680" y="391" fill="#8b98a5" font-size="8" text-anchor="middle">BEV perception</text>

  <!-- Key insight -->
  <rect x="25" y="410" width="710" height="26" rx="4" fill="#0b1220" stroke="#233043" stroke-width="0.8"/>
  <text x="380" y="427" fill="#fbbf24" font-size="9.5" text-anchor="middle">ViT proved that transformers beat CNNs at scale — the same attention mechanism works for text, images, and everything else.</text>

  <text x="380" y="460" fill="#6b7684" font-size="11" text-anchor="middle">The vision encoder in every frontier multimodal model is a ViT — patches are the visual equivalent of tokens.</text>
</svg>
```e Vision Transformer (ViT)** showed that the Transformer architecture built for language works just as well on images, and that insight is the bridge to today's multimodal models. Instead of processing pixels with convolutions, a ViT cuts an image into a grid of small patches, treats each patch as a token, and feeds the sequence into a standard Transformer encoder. Once an image is "just a sequence of tokens," it can share an architecture — and eventually a single model — with text, which is exactly what vision-language and multimodal systems exploit.\n\n```svg\n<svg viewBox="0 0 820 486" xmlns="http://www.w3.org/2000/svg" font-family="-apple-system,Segoe UI,Roboto,sans-serif">\n  <rect x="0" y="0" width="820" height="486" fill="#0d1117"/>\n  <text x="410" y="34" fill="#e6edf3" font-size="22" font-weight="700" text-anchor="middle">Vision Transformers &amp; Multimodal Models — Seeing with a Transformer</text>\n  <text x="410" y="58" fill="#8b98a5" font-size="13" text-anchor="middle">cut an image into patches, treat them as tokens — the same trick that lets one model jointly understand images and text</text>\n  <rect x="40" y="88" width="372" height="290" rx="8" fill="#0b1220" stroke="#233043" stroke-width="1.2"/>\n  <text x="58" y="114" fill="#cbd5e1" font-size="13" font-weight="700">ViT: an image becomes a sequence of tokens</text>\n  <rect x="64" y="132" width="20" height="20" fill="#1e3a5f" stroke="#38bdf8" stroke-width="0.8"/>\n  <rect x="86" y="132" width="20" height="20" fill="#2a4a6f" stroke="#38bdf8" stroke-width="0.8"/>\n  <rect x="108" y="132" width="20" height="20" fill="#1e3a5f" stroke="#38bdf8" stroke-width="0.8"/>\n  <rect x="64" y="154" width="20" height="20" fill="#2a4a6f" stroke="#38bdf8" stroke-width="0.8"/>\n  <rect x="86" y="154" width="20" height="20" fill="#38607f" stroke="#38bdf8" stroke-width="0.8"/>\n  <rect x="108" y="154" width="20" height="20" fill="#2a4a6f" stroke="#38bdf8" stroke-width="0.8"/>\n  <rect x="64" y="176" width="20" height="20" fill="#1e3a5f" stroke="#38bdf8" stroke-width="0.8"/>\n  <rect x="86" y="176" width="20" height="20" fill="#2a4a6f" stroke="#38bdf8" stroke-width="0.8"/>\n  <rect x="108" y="176" width="20" height="20" fill="#1e3a5f" stroke="#38bdf8" stroke-width="0.8"/>\n  <text x="96.0" y="214" fill="#7dd3fc" font-size="12" text-anchor="middle">split into patches</text>\n  <rect x="158" y="144" width="42" height="30" rx="5" fill="#0f1c17" stroke="#34d399" stroke-width="1.2"/>\n  <text x="179" y="164" fill="#6ee7b7" font-size="13" text-anchor="middle">CLS</text>\n  <rect x="205" y="144" width="42" height="30" rx="5" fill="#101826" stroke="#38bdf8" stroke-width="1.2"/>\n  <text x="226" y="164" fill="#7dd3fc" font-size="13" text-anchor="middle">1</text>\n  <rect x="252" y="144" width="42" height="30" rx="5" fill="#101826" stroke="#38bdf8" stroke-width="1.2"/>\n  <text x="273" y="164" fill="#7dd3fc" font-size="13" text-anchor="middle">2</text>\n  <rect x="299" y="144" width="42" height="30" rx="5" fill="#101826" stroke="#38bdf8" stroke-width="1.2"/>\n  <text x="320" y="164" fill="#7dd3fc" font-size="13" text-anchor="middle">3</text>\n  <rect x="346" y="144" width="42" height="30" rx="5" fill="#101826" stroke="#38bdf8" stroke-width="1.2"/>\n  <text x="367" y="164" fill="#7dd3fc" font-size="13" text-anchor="middle">4</text>\n  <text x="278" y="136" fill="#8b98a5" font-size="12" text-anchor="middle">patch tokens + a [CLS] token</text>\n  <line x1="132" y1="165.0" x2="152" y2="159" stroke="#38bdf8" stroke-width="1.6"/>\n  <path d="M145 165 L152 159 L143 158" stroke="#38bdf8" stroke-width="1.6" fill="none"/>\n  <rect x="64" y="198" width="324" height="28" rx="6" fill="#151b23" stroke="#6b7684" stroke-width="1.1"/>\n  <text x="226.0" y="216" fill="#c9d4e0" font-size="12" text-anchor="middle">+ linear embedding &#38; positional encoding</text>\n  <line x1="226.0" y1="174" x2="226.0" y2="196" stroke="#3f5169" stroke-width="1.6"/>\n  <path d="M222 188 L226 196 L230 188" stroke="#3f5169" stroke-width="1.6" fill="none"/>\n  <rect x="64" y="244" width="324" height="46" rx="8" fill="#2a1420" stroke="#fbbf24" stroke-width="1.6"/>\n  <text x="226.0" y="265" fill="#fcd34d" font-size="12.5" font-weight="700" text-anchor="middle">Transformer Encoder</text>\n  <text x="226.0" y="282" fill="#8b98a5" font-size="12" text-anchor="middle">self-attention lets every patch see every other patch</text>\n  <line x1="226.0" y1="226" x2="226.0" y2="242" stroke="#3f5169" stroke-width="1.6"/>\n  <path d="M222 234 L226 242 L230 234" stroke="#3f5169" stroke-width="1.6" fill="none"/>\n  <rect x="130" y="314" width="192" height="40" rx="8" fill="#0f1c17" stroke="#34d399" stroke-width="1.6"/>\n  <text x="226.0" y="332" fill="#6ee7b7" font-size="13" font-weight="700" text-anchor="middle">class label / image features</text>\n  <text x="226.0" y="347" fill="#8b98a5" font-size="13" text-anchor="middle">no convolutions — but needs large-scale pre-training</text>\n  <line x1="226.0" y1="290" x2="226.0" y2="312" stroke="#34d399" stroke-width="1.6"/>\n  <path d="M222 304 L226 312 L230 304" stroke="#34d399" stroke-width="1.6" fill="none"/>\n  <rect x="432" y="88" width="348" height="290" rx="8" fill="#0b1220" stroke="#233043" stroke-width="1.2"/>\n  <text x="450" y="114" fill="#cbd5e1" font-size="13" font-weight="700">From image tokens to multimodal</text>\n  <text x="452" y="136" fill="#93c5fd" font-size="12" font-weight="700" transform="rotate(-90 462 172)">image emb</text>\n  <rect x="502" y="132" width="18" height="18" fill="#0f2a1e" stroke="#34d399" stroke-width="1"/>\n  <text x="511.0" y="145" fill="#6ee7b7" font-size="13" text-anchor="middle">&#10003;</text>\n  <rect x="522" y="132" width="18" height="18" fill="#141b23" stroke="#2b3a4d" stroke-width="1"/>\n  <rect x="542" y="132" width="18" height="18" fill="#141b23" stroke="#2b3a4d" stroke-width="1"/>\n  <rect x="562" y="132" width="18" height="18" fill="#141b23" stroke="#2b3a4d" stroke-width="1"/>\n  <rect x="502" y="152" width="18" height="18" fill="#141b23" stroke="#2b3a4d" stroke-width="1"/>\n  <rect x="522" y="152" width="18" height="18" fill="#0f2a1e" stroke="#34d399" stroke-width="1"/>\n  <text x="531.0" y="165" fill="#6ee7b7" font-size="13" text-anchor="middle">&#10003;</text>\n  <rect x="542" y="152" width="18" height="18" fill="#141b23" stroke="#2b3a4d" stroke-width="1"/>\n  <rect x="562" y="152" width="18" height="18" fill="#141b23" stroke="#2b3a4d" stroke-width="1"/>\n  <rect x="502" y="172" width="18" height="18" fill="#141b23" stroke="#2b3a4d" stroke-width="1"/>\n  <rect x="522" y="172" width="18" height="18" fill="#141b23" stroke="#2b3a4d" stroke-width="1"/>\n  <rect x="542" y="172" width="18" height="18" fill="#0f2a1e" stroke="#34d399" stroke-width="1"/>\n  <text x="551.0" y="185" fill="#6ee7b7" font-size="13" text-anchor="middle">&#10003;</text>\n  <rect x="562" y="172" width="18" height="18" fill="#141b23" stroke="#2b3a4d" stroke-width="1"/>\n  <rect x="502" y="192" width="18" height="18" fill="#141b23" stroke="#2b3a4d" stroke-width="1"/>\n  <rect x="522" y="192" width="18" height="18" fill="#141b23" stroke="#2b3a4d" stroke-width="1"/>\n  <rect x="542" y="192" width="18" height="18" fill="#141b23" stroke="#2b3a4d" stroke-width="1"/>\n  <rect x="562" y="192" width="18" height="18" fill="#0f2a1e" stroke="#34d399" stroke-width="1"/>\n  <text x="571.0" y="205" fill="#6ee7b7" font-size="13" text-anchor="middle">&#10003;</text>\n  <text x="542" y="227" fill="#93c5fd" font-size="12" text-anchor="middle">text emb</text>\n  <text x="618" y="146" fill="#cbd5e1" font-size="13" font-weight="700">CLIP</text>\n  <text x="618" y="164" fill="#8b98a5" font-size="12">contrastive training aligns</text>\n  <text x="618" y="178" fill="#8b98a5" font-size="12">matching image–text pairs</text>\n  <text x="618" y="192" fill="#8b98a5" font-size="12">on the diagonal → zero-shot</text>\n  <line x1="450" y1="238" x2="762" y2="238" stroke="#233043" stroke-width="1"/>\n  <text x="450" y="260" fill="#a78bfa" font-size="12" font-weight="700">VLM: give an LLM eyes</text>\n  <rect x="450" y="276" width="86" height="42" rx="8" fill="#101826" stroke="#38bdf8" stroke-width="1.6"/>\n  <text x="493" y="296" fill="#7dd3fc" font-size="12" text-anchor="middle">vision</text>\n  <text x="493" y="310" fill="#7dd3fc" font-size="12" text-anchor="middle">encoder (ViT)</text>\n  <rect x="552" y="276" width="60" height="42" rx="8" fill="#1a1226" stroke="#c084fc" stroke-width="1.6"/>\n  <text x="582" y="301" fill="#d8b4fe" font-size="12" text-anchor="middle">projector</text>\n  <rect x="628" y="276" width="60" height="42" rx="8" fill="#2a1420" stroke="#fbbf24" stroke-width="1.6"/>\n  <text x="658" y="301" fill="#fcd34d" font-size="13" font-weight="700" text-anchor="middle">LLM</text>\n  <rect x="704" y="276" width="58" height="42" rx="8" fill="#0f1c17" stroke="#34d399" stroke-width="1.6"/>\n  <text x="733" y="301" fill="#6ee7b7" font-size="12" text-anchor="middle">answer</text>\n  <line x1="536" y1="297" x2="552" y2="297" stroke="#3f5169" stroke-width="1.5"/>\n  <path d="M544 301 L552 297 L544 293" stroke="#3f5169" stroke-width="1.5" fill="none"/>\n  <line x1="612" y1="297" x2="628" y2="297" stroke="#3f5169" stroke-width="1.5"/>\n  <path d="M620 301 L628 297 L620 293" stroke="#3f5169" stroke-width="1.5" fill="none"/>\n  <line x1="688" y1="297" x2="704" y2="297" stroke="#3f5169" stroke-width="1.5"/>\n  <path d="M696 301 L704 297 L696 293" stroke="#3f5169" stroke-width="1.5" fill="none"/>\n  <rect x="612" y="332" width="92" height="24" rx="5" fill="#151b23" stroke="#6b7684" stroke-width="1"/>\n  <text x="658" y="348" fill="#c9d4e0" font-size="12" text-anchor="middle">text prompt</text>\n  <line x1="658" y1="332" x2="658" y2="320" stroke="#3f5169" stroke-width="1.4"/>\n  <path d="M662 328 L658 320 L654 328" stroke="#3f5169" stroke-width="1.4" fill="none"/>\n  <rect x="40" y="392" width="740" height="70" rx="8" fill="#0d1117" stroke="#38bdf8" stroke-width="1.3"/>\n  <text x="58" y="420" fill="#7dd3fc" font-size="12.5" font-weight="700">One unifying idea</text>\n  <text x="200" y="420" fill="#e6edf3" font-size="12.5">image patches, words — even audio frames — all become tokens in one Transformer.</text>\n  <text x="58" y="444" fill="#8b98a5" font-size="12">That shared token space is why a single architecture can see, read, and — in “omni” models — map any modality to any other.</text>\n</svg>\n```\n\n**A ViT turns an image into patch tokens.** The image is split into fixed-size patches (often 16×16 pixels), each patch is flattened and linearly projected into an embedding, and learned positional encodings are added so the model knows where each patch sat. A special classification token is prepended, the whole sequence runs through Transformer encoder layers where self-attention lets every patch attend to every other, and the output at the classification token is used to predict the label. There are no convolutions anywhere in the core model.\n\n**ViT trades inductive bias for scale.** Convolutional networks bake in helpful assumptions — locality and translation equivariance — that ViTs lack, so on small datasets a ViT actually underperforms a comparable CNN. Its advantage appears with scale: pre-trained on very large image collections, a ViT matches or beats the best CNNs, because attention can learn flexible, long-range relationships that convolutions cannot. Data-efficient training recipes and distillation later narrowed the data requirement.\n\n**CLIP aligns vision and language in a shared space.** Trained contrastively on hundreds of millions of image–caption pairs, CLIP pairs an image encoder (usually a ViT) with a text encoder and pushes matching image–text embeddings together while pushing mismatched ones apart. The result is a joint embedding space where an image and its description land near each other, enabling zero-shot classification and image–text retrieval without task-specific training. CLIP's image encoder became the visual front-end for much of what followed.\n\n**Vision-language models give a language model eyes.** Systems such as LLaVA, Flamingo, and GPT-4V connect a pretrained vision encoder to a large language model through a small projection or adapter, so image-derived tokens enter the LLM's context alongside the text prompt. The LLM can then answer questions about a picture, read documents, or describe scenes. "Omni" or any-to-any models push this further, mapping among text, images, audio, and video within one model, so a single system can both perceive and generate across modalities.\n\n**The payoff and the open problems.** Tokenizing every modality unifies perception and language under one Transformer, which is why progress in one area now lifts the others, and why frontier assistants are natively multimodal. The hard parts are the cost of high-resolution and video inputs, hallucination on fine visual detail, and the resolution-versus-token-count trade-off — more patches mean sharper vision but a longer, more expensive sequence. Better visual tokenization and grounding are where much of the current research sits.\n\n| Stage | What it does | Key idea |\n|---|---|---|\n| Vision Transformer | image → patch tokens → encoder | patches are tokens |\n| CLIP | align image and text embeddings | one contrastive shared space |\n| Vision-language model | vision encoder feeds an LLM | image tokens in the LLM's context |\n| Omni / any-to-any | map among many modalities | one model perceives and generates |\n\nRead vision transformers and multimodal models through a *tokenize-everything* lens rather than a *new-vision-network* lens: the breakthrough is not a better image classifier but the realization that once patches, words, and audio frames are all tokens, one Transformer can attend across them — turning separate vision and language systems into a single model that sees and reads at once.\n