Heterogeneous Computing is the programming paradigm that leverages multiple types of processing units (CPUs, GPUs, FPGAs, NPUs, DSPs) within a single system to execute each portion of a workload on the processor architecture best suited for it — achieving higher performance and energy efficiency than any homogeneous approach.

Heterogeneous Architectures:

• CPU+GPU: most common heterogeneous configuration — CPU handles control-heavy, latency-sensitive tasks (OS, I/O, branching logic) while GPU handles data-parallel, throughput-oriented tasks (matrix math, image processing, neural network inference)
• CPU+FPGA: FPGA provides reconfigurable hardware acceleration for specific algorithms — achieves near-ASIC performance with post-deployment reprogrammability; Intel/AMD integrate FPGA fabric on server platforms
• CPU+NPU/TPU: dedicated neural processing units optimized for matrix multiply and convolution — fixed-function hardware achieves 10-100× better perf/watt than GPU for inference workloads
• Integrated SoCs: mobile and embedded SoCs integrate CPU, GPU, DSP, ISP, and NPU on a single die — Apple M-series, Qualcomm Snapdragon, and NVIDIA Orin exemplify this approach

Programming Frameworks:

• CUDA: NVIDIA-specific GPU programming model — maximum performance on NVIDIA hardware with rich ecosystem of libraries (cuBLAS, cuDNN, Thrust) and tools (Nsight, nvprof)
• OpenCL: open standard for heterogeneous computing across CPUs, GPUs, FPGAs — portable but often lower performance than vendor-specific solutions due to abstraction overhead
• SYCL/oneAPI: modern C++ abstraction over heterogeneous backends — Intel oneAPI targets CPU+GPU+FPGA with single-source programming and automatic device selection
• HIP: AMD's GPU programming model with near-identical syntax to CUDA — enables porting CUDA code to AMD GPUs with minimal changes; ROCm ecosystem provides equivalent libraries

Memory Management Challenges:

• Discrete vs. Unified Memory: discrete GPUs have separate memory requiring explicit data transfers (cudaMemcpy) — unified memory (CUDA managed memory, CXL-attached memory) provides automatic migration but with potential performance penalty from page faults
• Memory Coherency: CPU and GPU caches may not be coherent — explicit synchronization required after GPU kernel completion before CPU reads results; AMD APUs and CXL-connected accelerators provide hardware coherency
• Data Placement: optimal performance requires data to reside in the memory closest to the computing unit — NUMA-like effects between CPU DRAM, GPU HBM, and shared memory require careful data placement strategy

Heterogeneous computing represents the dominant paradigm for modern high-performance and energy-efficient computing — as Moore's Law slows, the primary path to continued performance improvement is through specialized accelerators, making heterogeneous programming skills essential for every performance-oriented developer.