Heterogeneous Computing is the system architecture paradigm that combines different types of processors — CPUs, GPUs, FPGAs, DSPs, and custom accelerators — within a single system, routing each portion of a workload to the processor type best suited for it, to achieve performance and energy efficiency impossible with any single processor type alone.

Why Homogeneous Systems Are Insufficient

CPUs excel at serial, branch-heavy, latency-sensitive code but waste power on massively parallel, regular workloads. GPUs provide 10-100x throughput for data-parallel work but perform poorly on serial, irregular code. FPGAs offer custom datapaths for specific algorithms. No single architecture is optimal for all workloads — heterogeneous systems assign each computation to the optimal accelerator.

Common Heterogeneous Configurations

• CPU + GPU: The dominant configuration for HPC, AI/ML, and graphics. The CPU handles OS, I/O, orchestration, and serial code. The GPU handles parallel computation (matrix multiply, convolution, simulation). The programming model: CPU launches GPU kernels, manages data transfers, and synchronizes results.
• CPU + FPGA: Used in network processing (SmartNICs), financial trading (ultra-low-latency inference), and genomics (custom alignment accelerators). FPGAs provide fixed-function throughput at lower power than GPUs for specific algorithms.
• CPU + Custom ASIC: Google TPU (tensor processing), Apple Neural Engine, AWS Graviton with Inferentia. Purpose-built silicon delivers the highest performance-per-watt for specific workloads but has zero flexibility for other tasks.
• APU / SoC Integration: AMD APU (CPU + GPU on one die), Apple M-series (CPU + GPU + Neural Engine + media engines), mobile SoCs (CPU + GPU + DSP + ISP + NPU). Shared memory eliminates copy overhead.

Programming Challenges

• Data Movement: Transferring data between CPU and accelerator memory is often the dominant cost. PCIe 5.0 provides 64 GB/s — fast but orders of magnitude slower than either processor's internal bandwidth. Unified memory (CUDA Unified Memory, HSA) automates page migration but cannot eliminate the physical transfer time.
• Task Partitioning: Deciding which code runs on which processor requires understanding each workload's characteristics (parallelism, memory access pattern, branch behavior). Poor partitioning wastes the accelerator's capability.
• Synchronization: Coordinating work between asynchronous processors with different clock domains, different memory spaces, and different completion times adds complexity not present in homogeneous systems.

Unified Memory Architectures

AMD's HSA (Heterogeneous System Architecture) and Apple's unified memory provide a single address space shared by CPU and GPU — eliminating explicit data copies. The hardware coherence protocol manages migration and caching. This dramatically simplifies programming at the cost of some hardware complexity.

Heterogeneous Computing is the pragmatic recognition that no single processor architecture can be best at everything — and that the highest performance comes from composing the right mix of specialized processors, connected by fast enough links, with software smart enough to use each one for what it does best.