deep learning
**Deep learning** is the subfield of machine learning that uses neural networks with many layers (deep architectures) to learn hierarchical representations of data — automatically discovering features from raw inputs (pixels, tokens, audio samples) without manual feature engineering. Deep learning is the engine behind every large language model (GPT, Claude, Gemini), every image generator (Stable Diffusion, DALL-E), every speech recognizer, and every recommendation system at scale. It is the workload that drives the entire AI chip industry: NVIDIA's datacenter revenue, Google's TPU program, and the global demand for HBM memory all exist because deep learning needs compute.
**Why "deep" matters — hierarchical feature learning.** A shallow model (linear regression, SVM, single-layer network) requires hand-crafted features. A deep network stacks many nonlinear layers, each learning progressively more abstract representations: early layers detect edges/n-grams, middle layers detect textures/phrases, deep layers detect objects/concepts. This automatic hierarchy is what allows a single architecture (the Transformer) to learn language, vision, code, and multimodal tasks from raw data — given enough parameters and compute.
**The computational structure of deep learning — why it needs AI chips:**
| Operation | % of training FLOPs | Hardware requirement | Chip response |
|---|---|---|---|
| Matrix multiply (GEMM) | 70–85% | Dense parallel arithmetic (TOPS) | Systolic arrays, tensor cores |
| Activation / normalization | 5–10% | Element-wise ops, memory bandwidth | Vector units, fused kernels |
| Attention (self/cross) | 10–20% (Transformers) | Quadratic memory, tiled compute | FlashAttention, HBM bandwidth |
| Gradient all-reduce (distributed) | Communication overhead | Inter-node bandwidth | NVLink, InfiniBand, UCIe |
| Data loading / preprocessing | I/O bound | Storage bandwidth, CPU | NVMe SSDs, DMA engines |
| Optimizer step (Adam, etc.) | 3–5% | Memory bandwidth (read/update params) | HBM capacity + BW |
**The deep learning stack — from math to silicon:**
- **Algorithms:** Transformer (attention + FFN), CNN, RNN/LSTM, diffusion, GAN
- **Frameworks:** PyTorch, JAX, TensorFlow — define computation graphs, auto-differentiate
- **Compilers:** XLA, TorchInductor, Triton — lower graphs to hardware-specific kernels
- **Runtime:** CUDA, ROCm, oneAPI — dispatch kernels to accelerators
- **Hardware:** GPU (NVIDIA H100/B200), TPU, custom ASIC — execute dense matmuls at 1000+ TFLOPS
- **Memory:** HBM3E (3–8 TB/s bandwidth) — feeds the compute units
- **Interconnect:** NVLink (900 GB/s), InfiniBand (400 Gb/s) — scales across chips/nodes
**Scaling laws — more compute, more data, more parameters = better.** Deep learning follows empirical power laws (Chinchilla, Kaplan et al.): model loss decreases predictably as a function of training compute (FLOPs), model size (parameters), and data volume (tokens). This means better AI = more hardware, driving an exponential growth in compute demand (~4× per year for frontier models). The entire AI chip industry — from NVIDIA's roadmap to TSMC's CoWoS capacity — is shaped by these scaling laws.
**Training vs inference — different hardware needs:**
| Aspect | Training | Inference |
|---|---|---|
| Precision | FP32/BF16/FP8 (mixed) | INT8/FP8/INT4 (quantized) |
| Batch size | Large (thousands) | Small (1–64) |
| Bottleneck | Compute (FLOPS) | Memory bandwidth (KV-cache reads) |
| Parallelism | Data + tensor + pipeline + expert | Tensor + batch only |
| Latency requirement | None (hours/days acceptable) | Strict (ms per token for chat) |
| Cost driver | GPU-hours × electricity | Tokens-per-second per dollar |
| Hardware | H100/B200 clusters, 8+ GPUs per node | Single GPU, or inference-optimized ASIC |
**Key deep learning architectures and their hardware implications:**
- **Transformer** (GPT, BERT, Llama): dense GEMM + attention → needs massive parallel FLOPs + HBM BW. See CFS transformer-architecture keyword.
- **CNN** (ResNet, EfficientNet): convolutions → can map to systolic arrays or Winograd transforms
- **Diffusion** (Stable Diffusion, DALL-E): iterative denoising → many sequential forward passes → latency-sensitive
- **MoE** (Mixtral, DeepSeek): sparse routing → needs all-to-all communication + large memory → see CFS mixture-of-experts keyword
- **Mamba/SSM**: linear recurrence → compute-bound, O(1) state → see CFS hybrid-attention-SSM keyword
```svg
```
**Deep learning and the CFS platform.** ChipFoundryServices exists because deep learning creates insatiable demand for better chips. The Inference Simulator (/infer) models LLM serving throughput. The Systolic-Array Simulator (/systolic) models the tensor-core compute that deep learning dominates. The HBM Simulator (/hbm) models the memory bandwidth that feeds those arrays. The FlashAttention Simulator (/flashattention) models the kernel that makes long-context attention practical. The KV-Cache Simulator (/kvcache) models the memory footprint of autoregressive generation. Together they cover the hardware stack that deep learning demands — from individual matmuls to datacenter-scale training clusters.