Deep learning | ChipFoundryServices

Home› Knowledge Base› Deep learning

Deep learning is the subfield of machine learning that uses neural networks with many layers (deep architectures) to learn hierarchical representations of data — automatically discovering features from raw inputs (pixels, tokens, audio samples) without manual feature engineering. Deep learning is the engine behind every large language model (GPT, Claude, Gemini), every image generator (Stable Diffusion, DALL-E), every speech recognizer, and every recommendation system at scale. It is the workload that drives the entire AI chip industry: NVIDIA's datacenter revenue, Google's TPU program, and the global demand for HBM memory all exist because deep learning needs compute.

Why "deep" matters — hierarchical feature learning. A shallow model (linear regression, SVM, single-layer network) requires hand-crafted features. A deep network stacks many nonlinear layers, each learning progressively more abstract representations: early layers detect edges/n-grams, middle layers detect textures/phrases, deep layers detect objects/concepts. This automatic hierarchy is what allows a single architecture (the Transformer) to learn language, vision, code, and multimodal tasks from raw data — given enough parameters and compute.

The computational structure of deep learning — why it needs AI chips:

Operation	% of training FLOPs	Hardware requirement	Chip response
Matrix multiply (GEMM)	70–85%	Dense parallel arithmetic (TOPS)	Systolic arrays, tensor cores
Activation / normalization	5–10%	Element-wise ops, memory bandwidth	Vector units, fused kernels
Attention (self/cross)	10–20% (Transformers)	Quadratic memory, tiled compute	FlashAttention, HBM bandwidth
Gradient all-reduce (distributed)	Communication overhead	Inter-node bandwidth	NVLink, InfiniBand, UCIe
Data loading / preprocessing	I/O bound	Storage bandwidth, CPU	NVMe SSDs, DMA engines
Optimizer step (Adam, etc.)	3–5%	Memory bandwidth (read/update params)	HBM capacity + BW

The deep learning stack — from math to silicon:

Algorithms: Transformer (attention + FFN), CNN, RNN/LSTM, diffusion, GAN
Frameworks: PyTorch, JAX, TensorFlow — define computation graphs, auto-differentiate
Compilers: XLA, TorchInductor, Triton — lower graphs to hardware-specific kernels
Runtime: CUDA, ROCm, oneAPI — dispatch kernels to accelerators
Hardware: GPU (NVIDIA H100/B200), TPU, custom ASIC — execute dense matmuls at 1000+ TFLOPS
Memory: HBM3E (3–8 TB/s bandwidth) — feeds the compute units
Interconnect: NVLink (900 GB/s), InfiniBand (400 Gb/s) — scales across chips/nodes

Scaling laws — more compute, more data, more parameters = better. Deep learning follows empirical power laws (Chinchilla, Kaplan et al.): model loss decreases predictably as a function of training compute (FLOPs), model size (parameters), and data volume (tokens). This means better AI = more hardware, driving an exponential growth in compute demand (~4× per year for frontier models). The entire AI chip industry — from NVIDIA's roadmap to TSMC's CoWoS capacity — is shaped by these scaling laws.

Training vs inference — different hardware needs:

Aspect	Training	Inference
Precision	FP32/BF16/FP8 (mixed)	INT8/FP8/INT4 (quantized)
Batch size	Large (thousands)	Small (1–64)
Bottleneck	Compute (FLOPS)	Memory bandwidth (KV-cache reads)
Parallelism	Data + tensor + pipeline + expert	Tensor + batch only
Latency requirement	None (hours/days acceptable)	Strict (ms per token for chat)
Cost driver	GPU-hours × electricity	Tokens-per-second per dollar
Hardware	H100/B200 clusters, 8+ GPUs per node	Single GPU, or inference-optimized ASIC

Key deep learning architectures and their hardware implications:

Transformer (GPT, BERT, Llama): dense GEMM + attention → needs massive parallel FLOPs + HBM BW. See CFS transformer-architecture keyword.
CNN (ResNet, EfficientNet): convolutions → can map to systolic arrays or Winograd transforms
Diffusion (Stable Diffusion, DALL-E): iterative denoising → many sequential forward passes → latency-sensitive
MoE (Mixtral, DeepSeek): sparse routing → needs all-to-all communication + large memory → see CFS mixture-of-experts keyword
Mamba/SSM: linear recurrence → compute-bound, O(1) state → see CFS hybrid-attention-SSM keyword

<svg viewBox="0 0 760 470" xmlns="http://www.w3.org/2000/svg" font-family="-apple-system,Segoe UI,Roboto,sans-serif">
  <rect x="0" y="0" width="760" height="470" fill="#0d1117"/>
  <text x="380" y="30" fill="#e6edf3" font-size="21" font-weight="700" text-anchor="middle">Deep Learning — Hierarchical Feature Extraction</text>
  <text x="380" y="50" fill="#8b98a5" font-size="12" text-anchor="middle">each layer transforms raw signal into increasingly abstract representations</text>

  <!-- Input: raw pixels (grid) -->
  <rect x="28" y="90" width="68" height="68" rx="3" fill="#0b1220" stroke="#3a4453" stroke-width="1"/>
  <!-- pixel grid -->
  <g stroke="#3a4453" stroke-width="0.4" opacity="0.6">
    <line x1="28" y1="107" x2="96" y2="107"/><line x1="28" y1="124" x2="96" y2="124"/><line x1="28" y1="141" x2="96" y2="141"/>
    <line x1="45" y1="90" x2="45" y2="158"/><line x1="62" y1="90" x2="62" y2="158"/><line x1="79" y1="90" x2="79" y2="158"/>
  </g>
  <!-- some "activated" pixels -->
  <rect x="45" y="107" width="17" height="17" fill="#60a5fa" opacity="0.3"/>
  <rect x="62" y="107" width="17" height="17" fill="#60a5fa" opacity="0.5"/>
  <rect x="62" y="124" width="17" height="17" fill="#60a5fa" opacity="0.4"/>
  <rect x="45" y="124" width="17" height="17" fill="#60a5fa" opacity="0.2"/>
  <text x="62" y="175" fill="#7dd3fc" font-size="10.5" text-anchor="middle">raw pixels</text>
  <text x="62" y="187" fill="#5b7b9f" font-size="9.5" text-anchor="middle">224 x 224 x 3</text>

  <!-- Arrow -->
  <path d="M100,124 L128,124" fill="none" stroke="#a78bfa" stroke-width="1.5"/>
  <polygon points="126,121 132,124 126,127" fill="#a78bfa"/>

  <!-- Layer 1: Edge detectors -->
  <rect x="136" y="82" width="80" height="85" rx="4" fill="#0f1a2a" stroke="#a78bfa" stroke-width="1.2"/>
  <!-- edge filter patterns -->
  <g fill="none" stroke-width="1.5">
    <line x1="148" y1="98" x2="168" y2="98" stroke="#c4b5fd"/><!-- horizontal edge -->
    <line x1="158" y1="108" x2="158" y2="128" stroke="#c4b5fd"/><!-- vertical edge -->
    <line x1="148" y1="138" x2="168" y2="118" stroke="#c4b5fd"/><!-- diagonal -->
    <path d="M180,98 C186,98 186,108 180,108" stroke="#c4b5fd"/><!-- curve -->
    <line x1="180" y1="118" x2="200" y2="138" stroke="#c4b5fd"/><!-- diagonal 2 -->
    <line x1="180" y1="138" x2="200" y2="118" stroke="#c4b5fd"/><!-- X pattern -->
  </g>
  <text x="176" y="175" fill="#c4b5fd" font-size="10.5" text-anchor="middle">edges</text>
  <text x="176" y="187" fill="#8b98a5" font-size="9.5" text-anchor="middle">conv 3x3, 64ch</text>

  <!-- Arrow -->
  <path d="M220,124 L248,124" fill="none" stroke="#a78bfa" stroke-width="1.5"/>
  <polygon points="246,121 252,124 246,127" fill="#a78bfa"/>

  <!-- Layer 2: Textures -->
  <rect x="256" y="82" width="80" height="85" rx="4" fill="#0f1a2a" stroke="#a78bfa" stroke-width="1.2"/>
  <!-- texture patterns -->
  <g stroke="#e0913a" stroke-width="1" fill="none">
    <!-- checkerboard -->
    <rect x="268" y="96" width="6" height="6"/><rect x="280" y="96" width="6" height="6"/>
    <rect x="274" y="102" width="6" height="6"/><rect x="286" y="102" width="6" height="6"/>
    <!-- stripes -->
    <line x1="268" y1="116" x2="290" y2="116"/><line x1="268" y1="120" x2="290" y2="120"/><line x1="268" y1="124" x2="290" y2="124"/>
    <!-- dots/circles -->
    <circle cx="306" cy="100" r="3"/><circle cx="316" cy="100" r="3"/><circle cx="306" cy="110" r="3"/><circle cx="316" cy="110" r="3"/>
    <!-- waves -->
    <path d="M300,120 C306,116 312,124 318,120 C324,116 330,124 336,120"/>
  </g>
  <text x="296" y="175" fill="#e0913a" font-size="10.5" text-anchor="middle">textures</text>
  <text x="296" y="187" fill="#8b98a5" font-size="9.5" text-anchor="middle">conv 3x3, 128ch</text>

  <!-- Arrow -->
  <path d="M340,124 L368,124" fill="none" stroke="#a78bfa" stroke-width="1.5"/>
  <polygon points="366,121 372,124 366,127" fill="#a78bfa"/>

  <!-- Layer 3: Parts -->
  <rect x="376" y="82" width="80" height="85" rx="4" fill="#0f1a2a" stroke="#a78bfa" stroke-width="1.2"/>
  <!-- recognizable shapes: eye, wheel, ear -->
  <g stroke="#34d399" stroke-width="1.3" fill="none">
    <!-- eye shape -->
    <ellipse cx="400" cy="105" rx="10" ry="6"/><circle cx="400" cy="105" r="2.5" fill="#34d399"/>
    <!-- ear shape -->
    <path d="M420,96 C428,96 430,112 424,116 C420,119 416,112 420,96"/>
    <!-- wheel -->
    <circle cx="400" cy="135" r="8"/><line x1="400" y1="127" x2="400" y2="143"/><line x1="392" y1="135" x2="408" y2="135"/>
    <!-- nose shape -->
    <path d="M422,128 C426,128 428,136 424,138 C420,140 418,136 422,128"/>
  </g>
  <text x="416" y="175" fill="#34d399" font-size="10.5" text-anchor="middle">parts</text>
  <text x="416" y="187" fill="#8b98a5" font-size="9.5" text-anchor="middle">conv 3x3, 256ch</text>

  <!-- Arrow -->
  <path d="M460,124 L488,124" fill="none" stroke="#a78bfa" stroke-width="1.5"/>
  <polygon points="486,121 492,124 486,127" fill="#a78bfa"/>

  <!-- Layer 4: Objects -->
  <rect x="496" y="82" width="80" height="85" rx="4" fill="#0f1a2a" stroke="#a78bfa" stroke-width="1.2"/>
  <!-- recognizable objects: face, car -->
  <g stroke="#fbbf24" stroke-width="1.3" fill="none">
    <!-- face outline -->
    <circle cx="520" cy="108" r="12"/>
    <circle cx="515" cy="104" r="2" fill="#fbbf24"/><circle cx="525" cy="104" r="2" fill="#fbbf24"/>
    <path d="M515,113 C518,116 522,116 525,113"/>
    <!-- car shape -->
    <path d="M505,140 L510,132 L530,132 L535,140 Z"/>
    <circle cx="512" cy="143" r="3"/><circle cx="528" cy="143" r="3"/>
    <!-- cat -->
    <path d="M548,95 L552,88 L556,95 M548,95 L556,95" /><circle cx="552" cy="102" r="7"/>
    <circle cx="549" cy="100" r="1.5" fill="#fbbf24"/><circle cx="555" cy="100" r="1.5" fill="#fbbf24"/>
  </g>
  <text x="536" y="175" fill="#fbbf24" font-size="10.5" text-anchor="middle">objects</text>
  <text x="536" y="187" fill="#8b98a5" font-size="9.5" text-anchor="middle">conv 3x3, 512ch</text>

  <!-- Arrow to classification -->
  <path d="M580,124 L608,124" fill="none" stroke="#a78bfa" stroke-width="1.5"/>
  <polygon points="606,121 612,124 606,127" fill="#a78bfa"/>

  <!-- Final: classification output -->
  <rect x="616" y="88" width="115" height="70" rx="5" fill="#1a2e1a" stroke="#34d399" stroke-width="1.3"/>
  <text x="673" y="108" fill="#6ee7b7" font-size="11" text-anchor="middle" font-weight="600">softmax output</text>
  <text x="673" y="126" fill="#8b98a5" font-size="10">cat:   0.92</text>
  <text x="673" y="140" fill="#8b98a5" font-size="10">dog:   0.05</text>
  <text x="673" y="154" fill="#6b7684" font-size="10">car:   0.02</text>

  <!-- Depth annotation -->
  <text x="380" y="210" fill="#e6edf3" font-size="12" text-anchor="middle" font-weight="600">depth = compositionality — each layer builds on the one before it</text>

  <!-- The "deep" part: why depth matters -->
  <rect x="30" y="228" width="700" height="88" rx="6" fill="#0b1220" stroke="#233043" stroke-width="1"/>
  <text x="380" y="250" fill="#e6edf3" font-size="12" text-anchor="middle" font-weight="600">Why depth works</text>
  <text x="50" y="272" fill="#8b98a5" font-size="10.5">• Shallow net needs exponentially many neurons to represent what a deep net does with a few layers</text>
  <text x="50" y="289" fill="#8b98a5" font-size="10.5">• Each layer is a differentiable function — chain rule (backprop) trains the whole stack end-to-end</text>
  <text x="50" y="306" fill="#8b98a5" font-size="10.5">• Skip connections (ResNet) let gradients flow through 100+ layers without vanishing</text>

  <!-- Architecture family comparison -->
  <text x="380" y="340" fill="#e6edf3" font-size="12" text-anchor="middle" font-weight="600">Architecture families — same principle, different inductive bias</text>

  <!-- CNN -->
  <rect x="30" y="354" width="165" height="62" rx="5" fill="#0f1a2a" stroke="#60a5fa" stroke-width="1.1"/>
  <text x="112" y="372" fill="#93c5fd" font-size="11" text-anchor="middle" font-weight="600">CNN</text>
  <text x="112" y="388" fill="#8b98a5" font-size="10" text-anchor="middle">local spatial filters</text>
  <text x="112" y="403" fill="#6b7684" font-size="9.5" text-anchor="middle">ResNet, EfficientNet, ConvNeXt</text>

  <!-- Transformer -->
  <rect x="210" y="354" width="165" height="62" rx="5" fill="#0f1a2a" stroke="#a78bfa" stroke-width="1.1"/>
  <text x="292" y="372" fill="#c4b5fd" font-size="11" text-anchor="middle" font-weight="600">Transformer</text>
  <text x="292" y="388" fill="#8b98a5" font-size="10" text-anchor="middle">global self-attention</text>
  <text x="292" y="403" fill="#6b7684" font-size="9.5" text-anchor="middle">GPT, ViT, BERT, LLaMA</text>

  <!-- SSM -->
  <rect x="390" y="354" width="165" height="62" rx="5" fill="#0f1a2a" stroke="#34d399" stroke-width="1.1"/>
  <text x="472" y="372" fill="#6ee7b7" font-size="11" text-anchor="middle" font-weight="600">SSM / Mamba</text>
  <text x="472" y="388" fill="#8b98a5" font-size="10" text-anchor="middle">linear recurrence</text>
  <text x="472" y="403" fill="#6b7684" font-size="9.5" text-anchor="middle">Mamba, RWKV, Hyena</text>

  <!-- Diffusion -->
  <rect x="570" y="354" width="165" height="62" rx="5" fill="#0f1a2a" stroke="#fbbf24" stroke-width="1.1"/>
  <text x="652" y="372" fill="#fcd34d" font-size="11" text-anchor="middle" font-weight="600">Diffusion</text>
  <text x="652" y="388" fill="#8b98a5" font-size="10" text-anchor="middle">iterative denoising</text>
  <text x="652" y="403" fill="#6b7684" font-size="9.5" text-anchor="middle">Stable Diffusion, DALL-E 3</text>

  <!-- Scale annotation -->
  <text x="380" y="436" fill="#6b7684" font-size="10.5" text-anchor="middle">frontier models: 100B+ parameters, 10T+ tokens, 10k+ GPUs, months of training — all relying on depth + scale + data</text>

  <text x="380" y="460" fill="#6b7684" font-size="11" text-anchor="middle">Deep = many composed layers. Every 10x in depth + compute unlocks qualitatively new capabilities.</text>
</svg>

Deep learning and the CFS platform. ChipFoundryServices exists because deep learning creates insatiable demand for better chips. The Inference Simulator (/infer) models LLM serving throughput. The Systolic-Array Simulator (/systolic) models the tensor-core compute that deep learning dominates. The HBM Simulator (/hbm) models the memory bandwidth that feeds those arrays. The FlashAttention Simulator (/flashattention) models the kernel that makes long-context attention practical. The KV-Cache Simulator (/kvcache) models the memory footprint of autoregressive generation. Together they cover the hardware stack that deep learning demands — from individual matmuls to datacenter-scale training clusters.

deep learningdeep learning fundamentalsdeep neural networkneural network trainingai trainingml training

Explore 500+ Semiconductor & AI Topics

From EUV lithography to CUDA optimization — search the full knowledge base or chat with our AI assistant.

🔍 Search Topics 💬 Ask CFSGPT 📚 Browse All