heterogeneous computing cpu gpu fpga

**Heterogeneous Computing** is **the programming paradigm that leverages multiple types of processing units (CPUs, GPUs, FPGAs, NPUs, DSPs) within a single system to execute each portion of a workload on the processor architecture best suited for it — achieving higher performance and energy efficiency than any homogeneous approach**. **Heterogeneous Architectures:** - **CPU+GPU**: most common heterogeneous configuration — CPU handles control-heavy, latency-sensitive tasks (OS, I/O, branching logic) while GPU handles data-parallel, throughput-oriented tasks (matrix math, image processing, neural network inference) - **CPU+FPGA**: FPGA provides reconfigurable hardware acceleration for specific algorithms — achieves near-ASIC performance with post-deployment reprogrammability; Intel/AMD integrate FPGA fabric on server platforms - **CPU+NPU/TPU**: dedicated neural processing units optimized for matrix multiply and convolution — fixed-function hardware achieves 10-100× better perf/watt than GPU for inference workloads - **Integrated SoCs**: mobile and embedded SoCs integrate CPU, GPU, DSP, ISP, and NPU on a single die — Apple M-series, Qualcomm Snapdragon, and NVIDIA Orin exemplify this approach **Programming Frameworks:** - **CUDA**: NVIDIA-specific GPU programming model — maximum performance on NVIDIA hardware with rich ecosystem of libraries (cuBLAS, cuDNN, Thrust) and tools (Nsight, nvprof) - **OpenCL**: open standard for heterogeneous computing across CPUs, GPUs, FPGAs — portable but often lower performance than vendor-specific solutions due to abstraction overhead - **SYCL/oneAPI**: modern C++ abstraction over heterogeneous backends — Intel oneAPI targets CPU+GPU+FPGA with single-source programming and automatic device selection - **HIP**: AMD's GPU programming model with near-identical syntax to CUDA — enables porting CUDA code to AMD GPUs with minimal changes; ROCm ecosystem provides equivalent libraries **Memory Management Challenges:** - **Discrete vs. Unified Memory**: discrete GPUs have separate memory requiring explicit data transfers (cudaMemcpy) — unified memory (CUDA managed memory, CXL-attached memory) provides automatic migration but with potential performance penalty from page faults - **Memory Coherency**: CPU and GPU caches may not be coherent — explicit synchronization required after GPU kernel completion before CPU reads results; AMD APUs and CXL-connected accelerators provide hardware coherency - **Data Placement**: optimal performance requires data to reside in the memory closest to the computing unit — NUMA-like effects between CPU DRAM, GPU HBM, and shared memory require careful data placement strategy **Heterogeneous computing represents the dominant paradigm for modern high-performance and energy-efficient computing — as Moore's Law slows, the primary path to continued performance improvement is through specialized accelerators, making heterogeneous programming skills essential for every performance-oriented developer.**